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Abstract 

This  s-tudy  explored  how  the  productivity  of  a 
configuration  and  data  management  division  within  a  matrix 
organization  can  be  improved  and  subsequently  recommended 
strategies  for  increased  productivity.  A  modified  Wagner 
and  Morse  questionnaire  facilitated  data  collection. 
Information  and  ratings  were  gathered  through  personal 
interviews  with  the  configuration  and  data  managers  and 
their  respective  program  managers  (matched-pair)  concerning 
specialized  competence,  aptitude,  utilization  and 
importance.  Additionally,  this  study  identified  situational 
factors  which  may  serve  to  increase  a  configuration  and  data 
manager's  competence  rating. 

Significant  findings  of  this  research  were:  (1)  the 
configuration  and  data  manager  is  relatively  insignificant 
when  compared  to  other  functional  program  personnel;  (2) 
there  are  few  sufficiently  knowledgeable  configuration  and 
data  management  personnel;  (3)  the  program  manager  possesses 
an  inadequate  understanding  of  the  duties  performed  by  a 
configuration  and  data  manager;  and  (4)  utilization  of  a 
configuration  and  data  manager  is  unrelated  to  the 
individual's  competency  level. 
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Two  alternative  recommendations  were  proposed  for 
increased  productivity  and  efficiency  of  the  configuration 
and  data  management  area: 

The  first  recommendation  proposes  a  major  re-structuring 
of  the  existing  configuration  and  data  management  support 
structure  by  which  configuration  and  data  managers  would  be 
transferred  to  work  in  the  system  program  office. 

The  second  recommendation  proposes  that  the  current 
organizational  structure  in  which  the  configuration  and  data 
management  personnel  work  in  not  be  disrupted. 

Four  strategies  for  enhanced  productivity  are 
subsequently  proposed  to  assist  in  the  implementation  of 
either  recommendation. 


IMPORTANCE  AND  UTILIZATION  OF  SPECIALIZED  COMPETENCE 


WITHIN  A  MATRIX  ORGANIZATIONAL  ENVIRONMENT 

I.  Introduction 

The  Research  Goal 

The  goal  of  this  study  is  to  investigate  factors  within  a 
matrix  structured  organization  which  may,  either  by 
themselves  or  cumulatively,  decrease  organizational 
productivity.  Factors  and  relationships  to  be  investigated 
throughout  the  course  of  this  research  project  include  role 
ambiguity,  intrinsic  (self-perceived)  and  extrinsic  (other- 
perceived)  aptitude,  specialized  competence,  role 
importance,  and  consequential  role  utilization.  The  desired 
outcome  of  this  research  is  to  identify  potential  solutions 
to  the  problem  of  limited  productivity  and,  subsequently,  to 
recommend  strategies  for  enhanced  organizational 
productivity  within  a  matrixed  structured  environment. 
General  Problem 

Central  to  the  concept  of  system  acquisition  management 
is  the  designation  of  a  single  program  manager  whose 
responsibilities  include  both  technical  and  business  aspects 
of  the  acquisition  program.  The  program  manager  is 
responsible  for  assembling  a  support  team,  gathered  from 
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personnel  in  appropriate  functional  areas,  to  assist  ±n 
accomplishing  the  program  objectives.  Within  a  matrix 
organisation,  program  expertise  is  provided  by  specialized 
individuals  from  functional  divisions  of  the  organization. 
The  program  manager  is  given  the  authority  required  to 
fulfill  his  responsibilities  as  manager  and  is  ultimately 
held  accountable  for  accomplishing  the  total  management  task 
(23:1-2).  The  delegation  of  roles  and  associated 
responsibilities  to  functional  personnel  of  the  management 
team  is  at  the  discretion  of  the  program  manager.  Each 
functional  division,  therefore,  has  defined  roles  and 
critical  responsibilities  based  cn  the  divisional  expertise. 
Previous  research  indicates  that  the  effectiveness  of  a 
program  manager  is  directly  related  to  the  collective 
competencies,  knowledge  and  skills  of  the  representative 
functional  division  team  members  (31).  Therefore,  it  can  be 
inferred  that  the  program  manager's  (extrinsic)  perception 
of  an  individual  functional  member's  specialized  competency 
will  be  based  on  the  demographics  (education,  experience, 
etc.)  related  to  competence  level  and  will  be  reflected  by 
the  degree  to  which  the  knowledge  and  skills  of  the 
functional  member  are  relied  upon  and  utilized  by  the 
program  manager.  However,  since  "the  chain  is  only  as 
strong  as  its  weakest  link",  the  tendency  of  many  program 
managers  to  use  their  own  "functional  competence"  instead  of 
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utilizing  a  functional  person  who  is  perceived  as  non- 
competent  may  not  strengthen  the  program  management  team  at 
all.  Thus,  the  wa j  a  program  manager  determines  functional 
competence  and/or  decides  to  utilize  a  functional  can  be 
critical  to  improving  the  effectiveness  of  program 
management . 

Providing  some  information  on  the  perceived  competence 
level  and  utilization  of  acquisition  personnel  is  a  study 
conducted  by  Captain  Miller  (1987).  Miller's  research 
identifies  competence  indicators  which  may  relate  to 
perceived  competence.  These  indicators  included  intrinsic 
[self-perceived]  aptitude,  formal  education,  program 
acquisition  phase  and  relative  importance  of  specialized 
competence  (24:8-10). 

Among  Miller's  competence  indicators  is  intrinsic 
aptitude.  For  the  purpose  of  this  research,  aptitude  may  be 
defined  as  an  individual's  ability  to  assimilate  new 
information  as  well  as  the  natural  tendency  to  personally 
engage  in  other  than  familiar  areas  and  issues.  Intrinsic 
aptitude,  according  to  his  conclusions,  may  have  a  direct 
positive  correlation  with  perceived  specialized  competence 
(24:8).  An  individual's  aptitude  may  be  measured  by 
determining  the  individual's  ability  and  readiness  to  learn 
new,  program-related,  technical  information  and  by  examining 
the  propensity  to  ask  questions  which  pertain  to  program 
issues  outside  their  functional  area. 
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Educational  status,  on  the  other  hand,  may  not  be  a  valid 
indication  of  an  individual's  intrinsic  abilities  or 
inclinations,  according  to  Miller's  research  conclusions. 
Contrary  to  Miller's  conclusions,  however,  previous  research 
by  Ekpo  concluded  that  the  higher  one's  educational  status, 
the  higher  the  self-estimate  he  makes  of  his  profession  of 
job  relevant  abilities  (13:411).  Assuming  that  an 
individual  must  assimilate  information  from  a  variety  of 
sources  including  personal  and  job  experiences,  in  addition 
to  formal  education,  it  may  be  that  cumulative  experiences 
rather  than  formal  education  alone,  may  serve  as  an 
indication  of  competency.  Whether  or  not  a  positive 
correlation  exists  between  an  individual's  educational 
status  and  perceived  competence  is  one  issue  to  be  addressed 
by  this  project. 

The  program  acquisition  phase,  Miller's  third  indication 
of  competence,  may  be  an  important  determining  factor  as  to 
whether  or  not  specialized  competence  is  required. 

Supporting  Miller's  conclusion  regarding  acquisition  phase 
is  research  conducted  by  Baumgarter  who  concludes  that  a 
high  level  of  competence  may  gain  critical  importance  as 
acquisition  programs  progress  through  later  phases  which 
involve  advanced  technology  (3:36).  Further  support  that 
program  phase  is  related  to  perceived  competence  is 
demonstrated  by  the  research  conclusions  of  Thamhain.  He 
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concludes  that  the  intensity  of  management  conflicts, 
especially  those  associated  with  programmatic  issues,  vary 
significantly  during  different  phases  of  an  acquisition 
program's  life  cycle  (31).  The  current  research  project 
will  explore  further  the  relationship  between  perceived 
specialized  competence  and  phase  of  an  acquisition  program, 
specifically  as  it  pertains  to  the  matrix-structured 
configuration  and  data  management  division. 

The  fourth  and  final  indication  of  competency  addressed 
in  Miller's  research  is  relative  importance  of  specialized 
competence.  The  relative  importance  of  an  individual 
functional  member's  belief  in  his  abilities  may  be  a 
critical  factor  in  the  utilization  of  the  functional  by  the 
program  manager  in  accomplishing  program  objectives.  If  a 
functional  projects  a  lack  of  knowledge,  relative  to  other 
program  functionals,  the  program  manager  will  also  rate  that 
functional's  specialized  competence  as  inadequate. 

Therefore,  utilization  of  the  functional  by  the  program 
manager  is  likely  to  be  minimal. 

Statement  of  the  Problem 

The  specific  aim  of  this  research  is  to  determine 
factors,  if  any,  which  may  contribute  to  decreased 
roductivitv  within  the  matrixed  configuration  and  data 
management  functional  office.  Productivity  will  be  defined 
as  the  efficient  utilization  of  resources.  Since  the 
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criteria  for  determining  efficiency  are  somewhat  subjective, 
the  organization's  commander  will  make  the  final 
determination  as  to  whether  or  not  a  resource  is  being 
utilized  to  an  optimum  level.  The  problem  of  decreased 
productivity  will  be  addressed  by  examining  aptitude, 
relative  importance,  the  perceived  specialized  competence  of 
the  conf iguration  and  data  manager,  the  utilization  of  this 
specialized  competence  by  the  respective  program  manager, 
and  the  organizational  formalization.  These  factors  are  all 
primary  elements  which  may  contribute  either  positively  or 
negatively  to  productivity  of  the  matrix  organization. 

Davis  stated  that,  in  the  acquisition  process  of  a  weapon 
system,  the  configuration  and  data  management  area  is 
perceived  as  " secondr ate "  from  the  program  manager's  point 
of  view  (9).  With  respect  to  programmatic  contributions, 
the  configuration  and  data  manager's  input  is  perceived  as 
less  important  than  that  of  other  functionals  including 
engineering,  logistics  and  program  control  (9).  These 
perceptions,  valid  or  not,  may  cause  a  decreased  utilization 
of  the  configuration  and  data  manager  in  the  development  of 
a  weapon  system.  While  it  can  be  expected  that  the 
treatment  a^orded  configuration  and  data  management 
functionals  will  reflect  these  perceptions,  decreased 
morale,  low  job  satisfaction,  employee  turnover,  etc.  can  be 
expected  to  result.  Such  associated  aspects  will  not, 
however,  be  addressed  in  this  research  project. 
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Importance  of  the  Study 


The  current  theme  for  a  majority  of  business-oriented 
organizations  (private  and  public)  is  to  "do  more  with 
less".  "In  order  to  stay  competitive,  a  business  must 
increase  productivity  10%  every  year"  (33).  Within  the 
Department  of  Defense  (DoD),  the  forecast  of  constant 
manpower  authorizations  has  sparked  concern  that  the 
productivity  and  effectiveness  of  the  DoD  must  increase  in 
order  to  accomplish  the  increasing  workload.  Within  ASD, 
this  productivity  concern  is  recognized  and  is  being 
addressed.  Further,  the  commander  at  ASD  realizes  that  the 
organization  cannot  operate  at  maximum  efficiency  if  any 
portion  of  the  organization  is  under-utilized.  This 
research  project  will  explore  the  perceived  competence, 
roles,  and  utilization  of  the  configuration  and  data 
manager.  Specifically,  this  research  will  attempt  to 
determine  means  by  which  the  productivity  of  the 
configuration  and  data  management  area  can  be  optimized? 
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Research  Objectives  of  the  Study 

In  dealing  with  the  problem  of  limited  productivity 
within  a  matrix  organization,  this  study  will  examine  five 
research  questions. 

Research  Question  1.  Is  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  related  to 
his/her  type  of  academic  education? 

Research  Question  2.  Is  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  related  to 
their  intrinsic  aptitude? 

Research  Question  3.  Is  the  perceived  importance  of  the 
configuration  and  data  manager's  specialized  competence 
related  to  the  specific  acquisition  phase  of  the  program? 

Research  Question  4.  Is  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  related  to 
the  degree  of  the  program  manager's  utilization  of  this 
specialized  competence? 

Research  Question  5.  Does  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  differ 
significantly  among  ASD  organizations? 
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Scope  of  this  Research 


Due  to  tile  fifteen-month  time  limitation  imposed  by  the 
Air  Force  Institute  of  Technology  (AFIT)  program,  this  study 
will  be  limited  to  quantifiable  subject  matter.  As  noted 
earlier,  qualitative  factors  of  functional  individuals  (such 
as  job  satisfaction,  retention/turnover  rates,  morale,  and 
rewards)  will  not  be  considered  in  this  study. 

Definitions 

Definitions  are  provided  in  Appendix  A.  These  definitions 
are  used  specifically  for  this  research  project. 

Assumptions 

The  following  assumptions  are  necessary  for 
implementation  of  this  study: 

1.  The  sample  groups  are  unbiased  and  independent. 

2.  The  sample  groups  are  representative  of  the  population 
from  which  they  are  drawn. 

3.  Survey  responses  and  interview  questions  are  addressed 
by  respondents  in  a  truthful  manner,  representative  of  the 
respondent's  sincere  feelings. 

4.  Organizational  units  to  be  evaluated  are  similar  in 
managerial  structure. 
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Employees  within  the  organizational  unit  are  generally 
located  in  the  same  physical  location,  report  to  the  same 
manager  and  perform  similar  tasks  (Figure  1)  (7:368). 

Advantages  of  the  functional  management  structure  include 
efficient  resource  usage,  advanced  skill  development, 
centralized  direction  and  decision  making,  and  excellent 
task  coordination  within  functions. 


Figure  1.  Functional  Structure  (37:47) 


Weaknesses  implicit  to  functional  structuring  include 
poor  task  coordination  across  functions,  slow  reaction  to 
changing  objectives,  lack  of  technological  innovation, 
limited  general  management  training,  and  difficulty  in 
recognizing  and  rewarding  quality  performance  (7:368). 

Given  the  diversified  technological  requirements  associated 
with  the  current  business  environment,  the  functional 
management  structure  is  not  the  optimum  organizational 
structure  for  most  situations  (32:1-2). 
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In  contrast  to 


the  functional  structure,  the  project  management  structure 
(self-contained),  contains  all  the  functional  units  required 
to  complete  a  specified  project  or  service  (Figure  2) 
(7:370).  The  advantages  of  the  project  management  structure 
include  the  ability  to  change  program  direction  with  a 
changing  environment,  immediate  access  to  resources, 
coordination  across  functional  units,  and  emphasis  on 
product  goals. 


r 


AOMIN 


1 - ^ 


RESEARCH 


Figure  2. 


Project  Structure  (37:47) 


Weaknesses  implicit  to  the  project  management  structure 
include  duplication  of  organizational  resources,  non¬ 
proficient  technical  specialization,  deficient  skill 
development,  poor  coordination  between  product  units,  and 
de-emphasi zed  top  management  decision  making  capability 
(7:375).  In  terms  of  access  to  resources  and  emphasis  on 
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product  goals,  this  management  structure  appears  optimum  to 
the  client;  however,  the  project  structure  may  not  be 
equally  beneficial  for  the  organization.  For  example, 
resource  duplication  may  not  be  an  appropriate  means  of 
achieving  increased  profits.  Therefore,  as  with  the 
functional  organization,  the  project  structural  approach  is 
not  an  appropriate  organizational  style  for  all 
applications . 

The  Matrix  Concept 

The  solution  then  must  be  an  optimum  blend  of  the  functional 

and  project  organizations.  The  hybrid  has  been  named  a 

matrix.  Davis  and  Lawrence,  notable  in  the  development  of 

matrix  management  theory  and  application,  describe  this 

management  structure  as  a  blend  of  the  traditional 

(functional)  and  behavioral  (project)  management  structures. 

Davis  and  Lawrence  define  a  matrix  structure  as: 

any  organization  that  employs  a  multiple  command  system 
that  includes  not  only  a  multiple  command  structure  but 
also  related  support  mechanisms  and  an  associated 
organization  culture  and  behavior  [8:3]. 

Expanding  on  the  matrix  concept  of  Davis  and  Lawrence, 

Moravec  states: 

.  .  .visualize  a  matrix  as  a  diamond-shaped  entity  with  the 

general  executive,  such  as  an  operations  manager  or  possibly 
a  division  head,  at  the  top;  matrix  managers  who  share 
common  subordinates  on  the  sides  of  the  diamond;  and  the 
subordinate  at  the  bottom.  As  the  balancing  of  this  diamond 
on  its  tip  suggests,  the  top  must  ensure  a  balance  of  the 
often  conflicting  objectives  of  the  managers  at  the  sides; 
otherwise,  the  balance  of  decisions  relating  to  the 
subordinate  at  the  bottom  is  impossible  [Figure  3]  [25:31]. 
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Operations  manager. 
Division  head, 
etc. 


The  significant  complication  of  the  matrix  concept  is  the 
two  distinct  "bosses  (functional  and  project)"  at  opposite 
but  equal  points.  The  "F-manager"  is  the  functional  manager 
and  the  "P-manager"  is  the  project  (program)  manager.  Each 
boss  possesses  different  objectives  and  goals.  According  to 
Cleland,  the  direct  result  of  matrix  dual  authority  is 
organizational  volatility.  "The  person  who  reports  to  two 
bosses  has  to  deal  with  two  sets  of  expectations,  two 
personalities,  and  two  kinds  of  impact  on  his  own 
priorities"  (6:31).  Working  within  a  matrix  management 
structure,  an  individual's  work  environment  is  filled  with 
conflicting  job-related  tasking,  one  from  the  program 
manager  and  one  from  his  functional  supervisor  (4:11).  An 
action  to  fulfill  one  boss's  needs  may  result  in 
confrontation  with  the  other  boss. 
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A  matrix  organization  may  be  as  simple  as  a  single 
project  unit  supported  by  the  functional  units,  as  displayed 
in  Figure  4.  Alternatively,  the  organization  may  be 
moderately  complex,  having  many  project  units  to  be 
supported  by  the  functional  units  (Figure  5).  Or,  the 
matrix  structure  may  be  extremely  complex,  as  illustrated  in 
Figure  6.  As  matrix  organizational  structure  complexity 
increases,  the  associated  problems  of  organizational 
authority,  role  definition  and  delegation  of  responsibility 
increase  proportionately. 
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Matrix  Structuring  Characteristics.  In  theory,  the 
matrix  organization  should  optimally  possess  the  positive 
organizational  advantages  of  both  previously  discussed 
structures  (functional  and  project),  without  the  negative 
attributes  of  either.  In  application,  however,  conflict  and 
communication  problems  are  associated  with  the  matrix 
organization  (7,29). 

Ma t r i x i ng  Goal s .  The  three  goals  of  matrix 
structuring,  according  to  Davis  and  Lawrence,  are: 

(1)  The  focusing  of  undivided  human  effort  on  two  (or 
more)  essential  organizational  tasks  simultaneously; 

(2)  the  processing  of  a  great  deal  of  information 

by  people  and  the  commitment  of  the  organization  to 
a  balanced,  reasoned  response  (a  general  manager 
response ) ,  and ; 

(3)  the  rapid  redeployment  of  human  resources  to  various 
projects,  products,  services,  clients,  or  markets 
[8:21}  . 

Following  definition  of  project  goals,  the  matrix-structured 
organization  must  incorporate  these  goals  into  the  overall 
business  strategy.  How  these  goals  are  incorporated  into 
the  business  strategy  and  ultimately  how  the  organization  is 
structured  depends  on  the  complexity  of  the  company's 
business  and  the  uncertainty  of  its  environment 
(19:89,8:70,15:37-38,22:5).  Within  the  Aeronautical  Systems 
Division,  goal  incorporation  and  organization  structuring 
are  established  by  the  commander.  A  representation  of  the 
strategy  to  meet  defined  objectives  is  shown  in  Figure  7. 
This  figure  reflects  the  commander's  perception  of  the 
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appropriately  tailored  matrix  structure  which  will 
efficiently  and  effectively  achieve  business  objectives. 

The  commander  does  have  the  authority  to  dynamically  change 
this  structure  if  necessary.  Figure  7  diagrammatically 
represents  the  current  ASD  functional  deputies:  program 
control  (AC),  engineering  (  EN )  ,  logistics  (AL),  and 
contracts  (PM).  Each  functional  deputy  is  central  to  the 
matrix  organization  and  physically  transfers  functional 
experts  to  the  system  program  offices.  Within  each  of  the 
three  system  program  offices  examined  in  this  research 
(ASD/AE,  ASD/RW,  and  ASD/YW),  a  secondary  matrix  structure 
is  employed  (Figure  8).  The  secondary  matrix  structure  is 
organized  with  appropriate  functional  disciplines  and 
personnel  supporting  specialized  system  program  offices.  In 
most  cases,  all  functional  personnel  work  together  in  one 
office  to  support  program  managers  in  other  physical 
locations . 
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Matrix  Structure  Assets.  The  advantages  of  a  matrix 
organization  are  the  combined  strengths  of  both  the 
functional  and  project  organizational  structures,  plus  the 
flexibility  inherent  to  the  matrix  concept  (2:41,28:22). 
According  to  Baber,  "This  (looseness)  implies  emphasis  on 
narrowly  defined  duties,  promoting  freer  communication,  and 
reducing  both  stratification  and  sectionalism  by  placing 
greater  emphasis  on  group  processes"  (2:41).  Davis  further 
states : 

The  matrix  idea  offers  the  potential  of  achieving  the 
flexibility  that  is  so  often  missing  in  conventional 
single-command  organizations  and  reconciling  this 
flexibility  with  the  coordination  and  economics  of  scale 
that  are  the  historic  strengths  of  large  organizations 
[8:70]  . 

In  application,  flexibility  functions  to  reduce  traditional 
confining  ooundaries,  thereby  allowing  less  rigid  role 
definition,  greater  productivity  and  greater  perceived 
proficiency.  Commenting  on  the  effectiveness  of  applying 
the  matrix  concept,  William  Goggin,  board  chairman  and  chief 
executive  officer  of  Dow  Corning,  states:  "We  perceive  our 
[matrix]  organization  to  be  a  dynamic  one;  and  to  date,  our 
experience  indicates  that  we  do  indeed  have  the  ability  to 
manage  change  rather  than  be  managed  by  it  (16:65). 

Matrix  Structure  Liabilities.  Conceptually, 
mafrixing  appears  to  be  the  optimum  organizational 
management  structure;  however,  the  flexibility  advantage  may 
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also  function  as  a  liability,  in  that  a  trade-off  is  made 

between  flexibility  and  role  standardization.  The  result  of 

this  exchange  is  role  ambiguity.  Total  role  ambiguity  is  a 

condition  in  which  employees  have  no  clear  understanding  of 

their  function  in  a  particular  organization  (20:167).  A 

characteristic  of  role  ambiguity  in  the  matrix  organization 

is  that  employees  have  overlapping  and  unclear 

responsibilities.  Thus  an  employee  has  no  knowledge  of 

when,  or  if,  his  part  of  the  task  is  completed  (35:73). 

This  research  effort  will  determine  whether  such  overlapping 

or  ambiguous  responsibilities  (role  ambiguity)  between 

configuration  and  data  managers  and  program  managers  are 

present  within  the  three  ASD  organizations  examined. 

The  relationship  between  role  ambiguity  and  matrix 

flexibility  is  expounded  upon  by  Joyce: 

Because  projects  end  and  new  ones  begin,  a  recurring 
process  of  change  is  inherent  in  matrix  structures. 
Adopting  a  matrix  structure  also  increases  role 
ambiguity.  In  such  arrangements,  responsibility  for  task 
execution  rests  with  both  lateral  and  hierarchical 
managers  making  explicit  definitions  of  their  respective 
responsibilities  difficult  [19:539]. 

In  an  effort  to  counter  the  role  ambiguity  factor,  Murray 

advocates  that  all  matrix  organizations  should  maintain  a 

conducive  environment  in  which  roles  are  clearly  defined  and 

understood  (26:251,5:19).  Although  Murray's  recommendation 

contradicts  the  flexibility  premise  of  a  matrix 

organization,  executing  such  a  recommendation  may  be 
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necessary  in  order  to  maintain  some  degree  of  consistency  in 

role  definition.  Studies  by  Kerr,  Jermier,  and  House  have 

shown  that  increased  organizational  formalization  reduces 

role  ambiguity  (27:822).  Adding  further  cor roboration , 

Podsakoff  states:  "One  possible  reason  for  the  negative 

relationship  between  organizational  formalization  and  role 

conflict  that  we  found  is  that  organizational  formalization 

may  adu  clarity  to  jobs"  ',27:8?6).  Th®  Aeronautical  sycl‘°n’c: 

Division  does  establish  some  basic  organizational 

formalization  in  application  of  the  matrix  concept.  This 

formalization  is  discussed  on  the  next  page  in  the  section 

of  this  thesis  titled,  Air  Force  Matrix  Application. 

History  of  Matrix  Structuring 

Matrix  structuring  was  first  applied  in  the  1960's  in  an 

attempt  to  resolve  inadequacies  which  had  become  evident  in 

project  and  functional  structuring.  According  to  Donald 

Kingdom,  formerly  of  TRW  Systems,  the  concept  of  a  matrix 

organization  was  created  for  government  contracts  (21:1). 

Kingdom  viewed  matrix  structuring  as: 

a  compromise  between  two  sets  of  needs:  the  customer's 
need  for  unified  direction  of  the  project  to  avoid  having 
to  negotiate  with  a  series  of  separate  functional 
managers,  and  the  company's  need  for  continuity  as  a 
viable,  developing  organization  building  up  its 
capability  to  handle  future  projects  as  well  as  current 
ones  through  the  existence  of  strong  specialist 
departments  [20:1]. 
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From  the  1960's  to  the  present,  the  matrix  management 
concept  has  been  utilized  in  all  facets  of  the  industrial 
environment.  Application  of  this  management  concept  was 
directed  towards  creating  an  organizational  structure  which 
would  allow  efficient  and  effective  accomplishment  of  the 
overall  objective. 

The  organizational  policies  of  the  Department  of  Defense 
do  not  correspond  with  the  overall  concept  of  matrix 
organizations.  The  Air  Force's  long-standing  policy  of 
creating  organizational  structures  with  centralized  control 
and  functional  groupings  complicates  the  complete 
[unrestricted]  application  of  the  matrix  concept  (11,17). 
Exceptions  to  the  Air  Force  policy  are  permitted,  however, 
which  allow  organizations  ".  .  .to  keep  pace  with 

technological  advances,  changing  missions,  and  concepts  of 
operation;  [and]  .  .  .to  streamline  the  decision-making 

process.  .  ."  (17:11).  Therefore,  owing  to  the  "exceptions" 

regulation,  the  Aeronautical  Systems  Division  (ASD)  is 
permitted  to  apply  the  matrix  management  concept. 

Air  Force  Matrix  Application 

As  stated  previously,  the  structure  of  an  organization 
must  complement  the  business  environment.  Currently,  most 
Air  Force  weapon  systems  produced  are  considered  highly 
technical  and  complex.  This  technical  complexity  has  forced 
the  Aeronautical  Systems  Division  to  adopt  a  matrix 


25 


organizational  structure.  ASD  Regulation  30-2  governs  the 
matrixing  of  personnel  within  ASD,  in  that  it  establishes 
the  degree  of  organizational  formalization  which  all 
organizations  must  obey  (12:1).  This  regulation  represents 
the  only  direction  imposed  by  the  ASD  commander.  Individual 
organizations  (system  program  office)  may  also  establish 
additional  formalization  guidelines.  However,  any 
additional  orgnr.i zational  formalization  guidance  must  be 
consistent  with  the  ASD  regulation. 

ASD  Matrix  Functional  Utilization.  A  key  concept  of 
matrix  management  theory  is  the  premise  that  all  the 
functional  support  will  be  balanced  (8:21).  In  the 
acquisition  environment,  if  the  program  manager  does  not 
fully  utilize  supporting  personnel,  then  he  may  not  b<_ 
perceived  to  be  as  competent  as  (and  probably  won't  be  as 
successful  as)  the  program  manager  who  recognizes 
specialized  competencies  and  subsequently  uses  all 
supporting  functional  personnel.  Functional  utilization  is 
determined  by  two  factors:  by  the  program  manager's 
perception  and  judgement  (knowledge  of  function)  of  the 
functional  and  by  the  program  manager's  clear  assignment  of 
the  functional's  roles  and  responsibilities.  If  the 
functional  is  perceived  as  lacking  competence  or  if  unclear 
role  assignments  (role  ambiguity)  lead  to  ineffective  task 
accomplishment,  the  utilization  of  the  particular  functional 
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will  be  noticeably  diminished  (35).  Whether  or  not  a 
program  manager  utilizes  a  functional  member  remains  the 
program  manager's  choice. 

Measurement  of  Competence 

The  effectiveness  of  a  program  manager  depends  upon  the 
competencies  of  functional  personnel  and  the  degree  to  which 
the  manager  utilizer  this  competence.  Determining  the 
competence  of  any  given  individual  within  any  professional 
area  is  difficult,  if  not  impossible.  Both  subjective  and 
objective  factors  are  involved  in  the  determination  of 
competence.  In  1975,  Wagner  and  Morse  developed  a  practical 
instrument  for  measurement  of  an  individual's  competence  for 
further  research  dealing  with  a  sense  of  competence.  The 
results  of  the  Wagner  and  Morse  effort  was  a  twenty-three 
item  questionnaire.  Questions  were  divided  into  four  major 
categories  of  factors.  The  first  factor  was  designated 
"Competence  Thema" .  This  factor  characterizes  the 
individual's  overall  global  feeling  of  a  sense  of 
competence.  The  second  factor,  designated  "Task 
Knowledge/Problem  Solving",  measures  the  ability  of  the 
respondent  to  understand  and  solve  problems  encountered  in 
the  work  environment.  Factor  three,  "Influence",  measures 
an  individual's  predisposition  toward  internal,  versus 
external,  control  in  making  job-related  decisions.  The 
final  factor,  that  of  "Confidence",  measures  the 
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respondent's  own  trust  and  faith  in  oneself,  which  is  an 
essential  part  of  a  person's  sense  of  competence  (34). 

The  reliability  obtainable  from  the  Wagner  and  Morse 
questionnaire  had  a  coefficient  of  0.96  ( Kude r-Ri cha rdson 
method)  on  the  test/re-test  group  (34:458).  Reliability  is 
a  measure  of  how  accurate,  on  the  average,  the  estimated 
scores  of  an  instrument  are.  Reliability  may  range  from 
0.00  (all  error  is  attributable  to  measurement),  to  1.00  (no 
error  is  due  to  measurement).  A  reliability  coefficient  of 
0.60  or  greater  is  considered  acceptable  for  repeatability. 

In  an  attempt  to  verify  the  23-item  questionnaire  of 
Wagner  and  Morse,  Synder  and  Morris  conducted  research  on 
the  four  designated  competence  factors.  Synder  and  Morris 
determined  that  three  of  the  four  established  factors 
provided  the  same  determination  or  measurement  of 
competence.  Therefore,  the  23-item  questionnaire  was 
reduced  to  15-items.  Competence  categories  were 
redesignated  as  sense  of  competence,  influence,  and  task 
knowledge.  The  reliability  coefficient  obtainable  from  the 
Synder  and  Morris  study  was  similar  to  that  obtainable  using 
the  Wagner  and  Morse  questionnaire  (30). 
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In  1985,  Lieutenant  Wilson  used  the  Wagner  and  Morse 
questionnaire  to  measure  the  sense  of  competence  of  Air 
Force  junior  officers  employed  in  civil  engineering. 
Tailoring  the  survey  instrument,  Lieutenant  Wilson  surveyed 
the  junior  officers  and  respective  supervisors  on  the  junior 
officers'  sense  of  competence.  The  reliability  coefficient 
of  Lieutenant  Wilson's  effort  was  0.85  (36). 

In  1987,  Captain  Miller  also  used  a  tailored  version  of 
the  Wagner  and  Morse  questionnaire  to  measure  the  sense  of 
technical  competence  among  Air  Force  senior  officers 
possessing  the  Air  Force  Specialty  Code  of  2916.  Captain 
Miller  used  the  matched-pair  methodology  approach,  gathering 
data  from  the  individual  Air  Force  officers  and  from  their 
respective  matched  technical  advisors  (program  engineers). 
The  reliability  coefficient  of  Captain  Miller's  measurement 
device  (0.82)  reconfirmed  the  excellent  repeatability  of  the 
Wagner  and  Morse  questionnaire  (24:53-55). 

A  tailored  modification  of  the  Wagner  and  Morse 
questionnaire  will  be  employed  for  data  collection  for  this 
research,  since  it  involves  the  complex  factors  of 
competence  determination,  and  since  the  results  need  to  be 
reliable.  This  methodology  is  the  most  reliable  means  of 
data  collection  presently  available.  Likewise,  a 
modification  of  the  Miller  approach  will  be  followed  to 
utilize  program  managers  to  corroborate  configuration  and 
data  managers  competence  ratings. 


29 


III.  Methodology- 

Introduction 

This  project  was  tailored  to  study  the  degree  to  which 
individual  configuration  and  data  managers  specialized 
competence  is  utilized  by  program  managers  within  an  ASD 
matrix  organization,  and  how  such  utilization  relates  to  the 
total  productivity  of  the  three  conf iguration  and  data 
management  divisions  surveyed.  The  design  of  this  study 
required  the  employment  of  personally  administered 
competence  questionnaires  for  data  collection.  Analysis  of 
data  was  accomplished  by  utilizing  one-way  analysis  of 
variances  and  Pearson  r  correlations. 

Justification 

A  survey  approach,  specifically  the  personal  interview 
technique,  was  chosen  as  a  data  collection  device  for  this 
research  project.  Other  techniques,  including 
experimentation  and  direct  observation/objective  measurement 
of  specialized  competence  were  deemed  inappropriate  due  to 
an  inability  to  control  actions  and  responses  of  human 
subjects  and  due  to  the  time  limitation  imposed  by  the  AFIT 
program. 

Surveying,  which  measures  an  individual's  perception  and 
opinions  regarding  a  specific  topic  may  be  of  two  types, 
mail  surveys  and  personal  interviews.  In  order  to 
effectively  address  the  research  question  posed  in  this 
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project,  personal  interviews  were  chosen  over  mail  surveys. 
The  personal  interview  facilitates  data  collection  in  that 
the  potential  misunderstanding  of  terms  and  constructs  used 
is  avoided.  The  personal  interview  is  a  "face-to-face", 
two-way  conversation,  initiated  by  an  interviewer  for  the 
purpose  of  obtaining  information  from  a  respondent  (14:160). 
The  greatest  advantage  of  this  data  collection  technique  is 
che  precise  nature  of  information  that  can  be  attained. 
Information  collected  by  interviewing  surpasses  that 
collected  by  the  mail  survey  in  that  the  researcher  can 
instantaneously  improve  question  quality  by  language 
modification,  concept/question  clarification  and  by  changing 
the  questioning  format.  Additionally,  validity  is  ensured 
since  ratings  by  program  managers  can  easily  be  matched  with 
respective  self-rating  of  the  configuration  and  data 
manage  r  s . 

Disadvantages  of  the  personal  interview  technique  include 
time  expenditure  of  both  the  interviewer  and  the  respondent, 
errors  resulting  from  inappropriate  statement  of  the 
question,  and  errors  in  response  transcription  by  the 
interviewer.  Most  damaging  to  the  validity  of  the  personal 
interview  technique  is  the  contribution  of  inaccurate 
information  by  the  respondent.  A  falsification  or 
inaccurate  presentation  of  information  may  substantially 
decrease  validity  of  data  and  subsequent  recommendations 
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(14:160).  However,  as  stated  in  chapter  one.  this  research 
assumes  that  all  information  is  true  and  correct. 
Construction  of  the  Data  Collection  (Measurement)  Devices 

To  ensure  that  information  pertaining  to  this  research 
project  was  gathered  in  a  thorough  and  systematic  manner, 
the  following  procedures  were  used  to  construct  of  the 
questionnaire  and  establish  the  format  of  the  personal 
inte  rviews . 

a.  Reviewed  the  literature  relevant  to  the  elements  of 
this  research  problem.  To  provide  a  foundation,  performed  a 
literature  review  summarizing  related,  previously  conducted 
studies  and  conclusions.  Data  collection  techniques  were 
assessed  and  evaluated  with  respect  to  this  research 
problem . 

b.  Constructed  a  preliminary  questionnaire  based  on  the 
proven  reliability  of  Captain  Miller's  adaptation  of  the 
Wagner  and  Morse  questionnaire. 

c.  Requested  a  critical  review  of  the  preliminary 
questionnaire  by  an  individual  knowledgeable  in  this  field 
(independent  of  this  research  project). 

d.  Revised  data  collection  instruments  for  enhanced 
effectiveness . 

e.  Tested  the  revised  survey  and  interview  questionnaire 
for  data  appropriateness  and  question  applicability  using  a 
panel  of  experts  from  the  research  field. 


f.  Revised  survey  and  interview  questions  using 


recommendations  provided  by  the  expert  panel. 

g.  Provided  a  copy  of  the  interview  questionnaire  to  the 
commanders  of  participating  organizations  to  obtain  their 
approval  of  the  use  of  the  survey  in  their  organization. 

h.  Collected  information  and  performed  data  analysis. 
Scope  of  the  Study 

The  population  of  interest  includes  configuration  and 
data  personnel  and  respective  program  managers  (matched 
pairs)  serving  in  System  Program  Offices  (SPO)  which  utilize 
a  matrix  management  structure  for  the  management  of 
individually  directed  programs.  The  cross-sectional  survey 
encompassed,  and  was  administered  to,  the  following 
Aeronautical  Systems  Division  organizations:  ASD./YW,  ASD/RW, 
and  ASD/AE.  Within  those  three  organizations,  a  sample  size 
of  27  of  the  29  configuration  and  data  managers  was 
determined  to  be  necessary  to  obtain  a  high  level  of 
external  validity  (10).  Since  each  configuration  and  data 
manager  supports  several  program  managers,  a  convenience 
sampling  technique  was  employed  in  selecting  the  program 
manager  pairings. 

Discussion  of  the  Questionnaire . 

This  research  project  utilized  the  framework  cf  Captain 
Miller's  interview  questions  with  modifications  to  obtain 
information  on  characteristics  unique  to  the  configuration 


and  data  manager.  The  questionnaire  employed  for  this  study 
provides  a  measure  of  the  level  of  perceived  competence  of 
the  configuration  and  data  manager.  Perceived  competence  is 
determined  by  arithmetic  summation  of  the  rating  values  for 
the  unique  characteristics  of  the  configuration  and  data 
management  function.  In  order  to  gather  information 
pertaining  to  the  unique  aspects  of  the  configuration  and 
data  management  area,  the  questionnaire  provides  a  means  for 
evaluation  of  roles  and  non-designated  managerial 
responsibilities . 

The  following  paragraphs  provide  an  explanation  of  each 
of  the  variables  used  in  this  research  effort.  Questions 
used  for  the  configuration  and  data  manager's  portion  of  the 
interview  are  found  in  Appendix  B.  Program  manager's 
questions  used  for  this  research  project  are  located  in 
Appendix  D. 

Configuration  and  Data  Manager's  Academic  Level . 

Question  1  of  the  configuration  and  data  manager's  survey 
addressed  education.  The  academic  level  of  the 
configuration  and  data  manager  was  based  either  on  the  level 
of  actual  formal  educational  completed  or  on  enrollment  in 
the  advanced  academic  education.  For  example,  if  an 
individual  possessed  a  high  school  diploma  and  was  pursuing 
an  associates  degree,  the  individual  was  assigned  the 
educational  status  of  an  associate. 
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Program  Acquisition  Phase.  Question  2.1  of  the 
configuration  and  data  manager's  survey  (Appendix  B), 
identified  the  program's  acquisition  phase.  The  program 
acquisition  phase  was  computed  from  the  program's  milestone 
approval  as  described  in  detail  in  DoD  Directive  5000.1  and 
DoD  Instruction  5uuu.2. 

Perceived  Configuration  and  Data  Manager's  Competence . 

The  competence  variable  was  derived  by  the  arithmetic 
summation  of  five  independent  areas  cited  in  questions  3.1 
through  3.5  of  the  configuration  and  data  manager's  survey 
and  in  questions  1.1  through  1.5  of  the  program  manager's 
survey.  Each  question  was  given  equal  weight  in  the 
determination  of  the  competence  variable.  Questions  3.2  - 
3.5  and  1.2  -  1.5  of  this  project  were  essentially  identical 
to  those  of  Miller.  However,  since  questions  3.1  and  1.1 
explored  the  critical  specialty  areas  of  the  configuration 
and  data  manager,  an  expansion  (delineation  of  important 
subtopics)  of  the  survey  framework  used  by  Miller  was 
required  to  thoroughly  evaluate  the  level  of  perceived 
competence  given  the  breadth  of  responsibilities  of  the 
configuration  and  data  manager.  Questions  3.1. a  through 
3.1.g  of  the  configuration  and  data  manager's  survey  and 
questions  1.1. a  through  1.1. g  of  the  program  manager's 
survey  addressed  unique  tasks  and  issues  relating  to 
configuration  and  data  management.  Configuration  and  data 
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managers  and  program  managers  were  asked  to  rate  their 
abilities  in  performing  specific  duties.  They  were  also 
asked  to  identify  which  duties  were  solely  configuration  and 
data  manager  responsibilities,  which  were  solely  program 
manager  responsibilities,  and  which  were  shared. 

Configuration  and  Data  Manager's  Utilization.  Question 
3.8  of  the  configuration  and  data  manager's  survey,  and 
question  1.8  of  the  program  manager's  survey,  addressed  the 
degree  to  which  the  program  manager  utilized  the  specialized 
skills  of  the  configuration  and  data  manager. 

Configuration  and  Data  Manager's  Intrinsic  Aptitude . 
Questions  4.1  and  4.2  of  the  configuration  and  data 
manager's  survey,  and  questions  2.1  and  2.2  of  the  program 
manager's  survey,  addressed  intrinsic  (self-perceived),  and 
extrinsic  (program  manager-perceived)  aptitude, 
respectively,  to  determine  the  individual's  aptitude  as  an 
indicator  of  competence.  These  questions  were  similar  to 
those  used  by  Captain  Miller  to  evaluate  aptitude.  The 
arithmetic  summation  of  these  two  equally  weighted  questions 
constituted  the  aptitude  variable. 

Configuration  and  Data  Management  importance.  Questions 
5.1  and  5.2  of  the  conf iguration  and  data  manager's  survey, 
as  well  as  questions  3.1  and  3.2  of  the  program  manager's 
survey,  addressed  the  importance  of  configuration  and  data 
management.  The  importance  of  possessing  a  high  level  of 
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configuration  and  data  management  competence,  as  it  relates 
to  program  acquisition  phase,  may  provide  an  indication  of 
when  the  configuration  and  data  manager's  involvement  with 
the  program  development  should  occur.  These  questions  were 
adapted  from  Captain  Miller's  survey  which  addressed  a 
similar  variable.  The  arithmetic  summation  of  these  two 
equally-weighted  questions  constituted  the  "importance" 
variable . 

Other  Information.  Questions  6.1  through  6.3  of  the 
configuration  and  data  manager's  survey  and  questions  4.1 
through  4.3  of  the  program  manager's  survey  explored  various 
subjects  directly  related  to  configuration  and  data 
management  and  to  program  management.  Configuration  and 
data  managers  and  respective  program  managers  were  asked  to 
comment  on  key  areas  including  training  needs  and 
deficiencies,  the  background  of  the  configuration  and  data 
manager,  and  the  areas  of  expertise  possessed  by  a  good 
configuration  and  data  manager  were  examined.  Analysis  of 
these  comments  consisted  of  numerical  tabulation  and 
frequency  distribution.  The  comments  were  compiled  and 
serve  as  individual  opinions  of  the  attributes  and  qualities 
deemed  necessary  in  a  proficient  configuration  and  data 
manager  (Appendix  G). 
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Data  Analysis 

The  following  sections  describe  the  data  analysis  methods 
used  to  collect  and  analyze  data  obtained  in  support  of  this 
research  project. 

Description  of  Research  Questions 

Research  Question  1.  Is  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  related  to 
his/her  type  of  academic  education? 

Statement  of  Hypothesis.  Specialized  competence  ratings 
of  configuration  and  data  managers  does  not  vary 
significantly  and  are  not  dependent  upon  the  educational 
background  of  the  manager. 

This  research  question  proposes  that  a  relationship  exists 
between  the  type  of  academic  educational  degree  obtained  and 
the  configuration  and  data  manager's  intrinsic  rating  and 
the  program  manager's  perceived  rating  of  specialized 
competence.  Both  an  independent  and  dependent  variable  are 
involved  in  analysis  of  this  hypothesis.  The  independent 
variable  was  the  conf iguration  and  data's  manager's  academic 
background.  This  variable  was  treated  as  a  categorical 
variable.  Academic  degrees  were  placed  into  five 
categories.  Separation  of  categories  was  based  solely  on 
relevant  course  curriculum  required  for  degree  completion. 
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The  categories  were: 

1.  Post-Bachelors  Degree. 

2.  Bachelors  Degree-Technical. 

3.  Bachelors  Degree-Non-Technical . 

4.  Associate  Degree. 

5.  High  School  Diploma. 

The  dependent  variables  in  the  analysis  of  the  first 
research  hypothesis  is  the  configuration  and  data  manager's 
self-perceived  (intrinsic)  and  program  manager's  perceived 
(extrinsic)  level  of  the  conf iguration  and  data  manager's 
specialized  competence.  These  intrinsic  and  extrinsic 
specialized  competence  variables  were  measured  using  a 
subset  from  the  Wagner  and  Morse  competence  questionnaire 
(questions  3.1  -  3.5  of  the  configuration  and  data  managers' 
survey  and  questions  1.1  -  1.5  of  the  program  managers' 
survey).  The  five  subset  items  were  modified  for  specific 
application  to  configuration  and  data  management  specialized 
competence.  The  scores  for  the  five  items  were  summed  and 
averaged  to  compute  a  composite  variable,  which  constituted 
the  dependent  variable  (competence  rating). 

Data  Analysis.  Unbalanced  single-factor  analysis  of 
variance  (ANOVA)  (alpha  level  =  0.05)  was  used  to  determine 
if  a  significant  difference  exists  between  values  of  the 
dependent  variables.  The  ANOVA  test  allows  determination  of 
significant  differences  between  categories.  An  ANOVA  test 
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is  the  calculation  of  the  squared  within-sample  variation 
divided  by  the  squared  between-sample  variation.  The 
resulting  value  may  be  converted  into  a  numeric  probability. 
The  alpha  probability  level  of  0.05  was  used  throughout  this 
research  project  as  the  criteria  of  acceptability.  Using  an 
alpha  level  of  0.05,  if  the  resulting  probability  (after  the 
ANOVA  calculations)  of  two  sample  populations  is  0.05  or 
less,  then  the  populations  are  considered  to  be 
significantly  different  from  each  other.  The  ANOVA  test  was 
performed  both  for  the  configuration  and  data  managers' 
self-perceived  (intrinsic)  competence  ratings  and  for  the 
program  managers'  extrinsic  competence  ratings. 

Research  Question  2.  Is  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  related  to 
their  intrinsic  aptitude? 

Statement  of  Hypothesis.  Extrinsic  and  intrinsic 
performance  ratings  of  a  configuration  and  data  managers 
specialized  competence  are  positively  correlated  with 
respective  ratings  of  aptitude. 

This  research  question  addresses  the  relationship  between 
the  configuration  and  data  manager's  aptitude  and  their 
specialized  competence,  both  as  stated  by  the  configuration 
and  data  manager  and  as  perceived  by  the  program  manager. 

The  independent  variable  is  the  perceived  aptitude  of  the 
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configuration  and  data  manager.  Aptitude  was  determined  by 
responses  to  questions  4.1  and  4.2  and  questions  2.1  and  2.2 
of  the  configuration  and  data  manager's  and  program 
manager's  survey,  respectively.  For  the  purpose  of  this 
study,  the  construct  of  aptitude  included  the  following 
characteristics: 

1.  Configuration  and  data  manager's  propensity  to  ask 
questions  pertaining  to  ambiguous  aspects  of  the  program. 

2.  Configuration  and  data  manager's  quickness  in 
assimilating  new  program  information. 

The  dependent  variable,  specialized  competence,  was  measured 
and  treated  as  described  in  Research  Question  1. 

Data  Analysis.  Two  sets  of  data  analyses  were  performed. 
One  analysis  addressed  the  intrinsic,  or  self-rating,  of  the 
configuration  and  data  manager's  aptitude  and  the 
specialized  competence.  The  second  data  analysis  performed 
addressed  the  extrinsic  rating  of  the  conf iguration  and  data 
manager's  aptitude  and  the  specialized  competence  as 
perceived  by  the  program  manager.  A  correlation  analysis 
was  performed  to  determine  the  Pearson  r  correlation 
coefficient  representing  the  degree  of  correlation  between 
the  conf iguration  and  data  managers'  specialized  competence 
and  their  specialized  aptitude.  The  Pearson  r  correlation 
ranges  in  value  from  -1.00  to  +1.00.  A  correlation 
coefficient  of  r  =  +1.00  signifies  a  perfect  linear 
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relationship.  As  applied  to  this  research,  a  correlation 
coefficient  of  +1.00  would  indicate  that  aptitude  directly 
improves  competence.  Likewise,  a  correlation  coefficient  of 
r  =  -1.00  indicates  a  perfect  negative  or  inverse 
relationship  between  the  two  variables  or,  restated,  that 
aptitude  decreases  competence.  A  correlation  coefficient  of 
r  =  0.00  suggests  there  is  no  relationship  between  the 
respective  values  of  the  two  variables.  The  correlation 
coefficient  r  is  calculated  by  summing  the  differences 
between  the  means  and  the  standard  deviation  of  paired 
values  of  specialized  competence  and  specialized  aptitude 
divided  by  the  sample  size  minus  one  (degrees  of  freedom). 

Research  Question  3.  Is  the  perceived  importance  of  the 
configuration  and  data  manager's  specialized  competence 
related  to  the  specific  acquisition  phase  of  the  program? 
Statement  of  Hypothesis.  The  perceived  importance  of 
proficient  specialized  competence  ratings  (extrinsic  and 
intrinsic)  of  a  configuration  and  data  manager  differs 
significantly  with  the  system  program  phase. 

Research  question  three  addresses  how  critical  or  important 
the  specialized  competence  of  the  configuration  and  data 
manager  is  during  various  system  program  phases.  The 
independent  variable  is  the  program  phase.  Program  phase 
was  determined  by  the  configuration  and  data  managers' 
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response  to  question  2.1  of  their  survey.  This  variable  was 
treated  as  a  categorical  variable  with  the  following  program 
phase  categories: 

1.  Concept  definition 

2.  Demonstration  and  Validation 

3.  Full-Scale  Development 

4.  Production  and  Deployment 

5.  Operations  and  Support 

If  the  phase  variable  was  not  definable,  due  to  concurrent 
program  phases,  the  configuration  and  data  manager 
preferentially  defined  the  phase  as  that  which  posed  the 
most  important  program  issues.  The  dependent  variable  is 
the  perceived  importance  of  the  proficiency  level  of  the 
configuration  and  data  manager's  specialized  competence. 

Data  Analysis.  An  unbalanced,  single-factor  analysis  of 
variance  (ANOVA)  (alpha  level  =  0.05)  was  used  to  determine 
significant  differences  between  dependent  variables. 

Results  of  the  ANOVA  test  allow  determination  of  significant 
differences  existing  between  program  phases.  Testing  was 
performed  twice;  once  using  the  configuration  and  data 
manager's  self-ratings  and  once  using  the  program  manager's 
extrinsic  ratings. 
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Research  Question  4.  Is  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  related  to 
the  degree  of  the  program  manager's  utilization  of  this 
specialized  competence? 

Statement  of  Hypothesis.  A  positive  correlation 
exists  between  the  overall  performance  rating  of 
a  configuration  and  data  manager's  specialized 
competence  and  the  program  manager's  utilization  of  that 
specialized  competence  for  accompl i shing  program 
objectives . 

Research  question  four  addresses  the  relationship  between 
the  configuration  and  data  manager's  specialized  competence 
rating  and  the  degree  to  which  the  program  manager  utilizes 
and  relies  upon  this  specialized  competence.  The  dependent 
variable  is  the  perceived  specialized  competence  of  the 
configuration  and  data  manager  (determined  for  research 
question  one).  The  independent  variable  is  utilization  of 
the  configuration  and  data  manager's  specialized  competence 
by  the  program  manager.  Utilization  was  determined  by 
responses  to  question  3.8  and  question  1.8  of  the 
configuration  and  data  manager's  and  program  manager's 
survey,  respectively. 

Data  Analysis.  Correlation  analysis  was  performed  using 
the  Pearson  r  correlation  coefficient  for  the  configuration 
and  data  manager's  specialized  competence  and  degree  of 
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utilization.  The  correlation  analysis  was  performed  twice; 
once  for  the  program  manager's  extrinsic  utilization  ratings 
of  the  configuration  and  data  manager's  specialized 
competence,  and  once  for  the  intrinsic  conf i gu ra t i on  and 
data  managers  utilization  of  specialized  competence  rating. 

Research  Question  5.  Does  the  perceived  specialized 
competence  of  the  configuration  and  data  manager  differ 
significantly  among  ASD  organizations? 

Statement  of  Hypothesis .  The  overall  performance  rating 
of  a  conf iguration  and  data  manager's  competence  does  not 
vary  among  ASD  organizations.  Such  lack  of  variation  may 
be  attributed  to  the  consistency  of  written  internal 
polices  of  a  given  organization. 

Research  question  five  addresses  the  relationship  between  a 
configuration  and  data  managers  specialized  competence 
ratings  (intrinsic  and  extrinsic)  and  the  internal  policies 
of  the  organization  subunits.  Central  to  this  research 
question  is  the  formalization  of  organizational  policies 
relating  to  these  specialized  functions.  Formalization,  as 
noted  in  chapter  2  of  this  research,  reduces  the  role 
ambiguity  factor.  The  dependent  variable  is  the 
configuration  and  data  manager's  perceived  specialized 
competence.  The  independent  variable  is  the  configuration 
and  data  manager's  responsible  organization.  This  variable 
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was  treated  as  a  categorical  variable  with  one  category  for 
each  of  the  three  organizations  surveyed. 

Data  Analysis.  An  unbalanced  single-factor  analysis  of 
variance  (ANOVA)  (alpha  level  =  0.05)  was  used  to  determine 
if  a  significant  difference  in  competence  ratings  exists 
between  the  three  organizations  surveyed.  Results  of  the 
ANOVA  test  will  highlight  any  significant  differences  in 
recognition  of  specialized  competence  between  organizations 
which  may  be  attributed  to  formalized  organizational 
policies.  Testing  will  be  performed  twice;  once  using  the 
conf iguration  and  data  manager's  intrinsic  ratings,  and  once 
using  the  program  manager's  extrinsic  ratings  of  the 
configuration  and  data  manager's  specialized  competence. 
Other  Information 

The  survey  instrument  (questions  6.1  -  6.3  of  the 
configuration  and  data  manager  survey  and  4.1  -  4.3  of  the 
program  manager  survey)  addressed  subject  matter  directly 
applicable  to  configuration  and  data  management  as  well  as 
the  essential  attributes  or  qualities  identified  with 
proficient  configuration  and  data  managers.  Information 
gathered  from  configuration  and  data  managers  and  program 
managers  was  used  to  determine  which  configuration  and  data 
manager's  roles  or  individual  attributes  were  key  indicators 
of  high  competence  levels.  Data  analysis  conducted  was  a 
simple  numerical  tabulation  of  the  configuration  and  data 
manager's  and  program  manager's  responses. 
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Reliability  and  Convergent  Validity 

In  order  to  promote  and  subsequently  assess  the 
reliability  and  validity  of  this  research  effort,  the 
following  criteria  were  addressed. 

Sample  Size.  For  this  research  project,  a  survey 
interview  sample  size  sufficient  to  achieve  a  95  percent 
( +/-  5  percent)  confidence  level  was  determined  to  be 
appropriate  for  achieving  a  statistically  sound 
representation  of  the  population  (10).  Based  on  a  finite 
population  of  29  total  configuration  and  data  managers  in 
three  organizations,  the  sample  size  was  determined  to  be 
27  . 

Measurement  Reliability.  The  determination  of  a  numeric 
value  of  many  hypothesis  variables  (e.g.,  intrinsic 
competence  rating)  was  based  the  average  of  at  least  two 
survey  instruments  items.  Cronbach  alpha  reliability 
coefficients  were  computed  for  each  of  these  composite 
variables  as  an  indication  of  the  measurement  reliability  of 
the  instruments.  The  Cronbach  alpha  reliability  coefficient 
is  designed  for  situations  where  the  goal  is  to  assess  the 
reliability  of  a  sum,  or  weighted  sum,  as  an  estimate  of  a 
variable  (e.g.,  questions  3.1  -  3.5  and  intrinsic  competence 
ratings) . 
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Convergent  Validity.  Convergent  validity  is  a  measure  of 


how  thoroughly  independent  instruments  measure  the  same 
construct,  or  restated,  of  how  similarly  separate  groups 
view  the  same  phenomenon.  In  order  to  maximize  the 
convergent  validity  for  this  research  effort,  the 
configuration  and  data  manager,  and  the  respective  program 
manager,  received  survey  instruments  which  contained 
identical  items  for  measuring  opinion/perception  variables. 
To  determine  if  there  was  a  significant  difference  between 
the  self-perceived  (intrinsic)  ratings  and  other-perceived 
(extrinsic)  ratings  of  all  variables,  paired  t-tests  were 
conducted.  The  paired  t-test  indicates  if  the  means  of  two 
groups  are  different,  where  the  samples  were  drawn  in  pairs. 
The  test  is  actually  evaluating  whether  the  mean  of  the 
differences  of  the  pairs  is  different  from  zero.  If  the 
means  were  significantly  different,  one  could  draw  the 
inference  that  the  questionnaire  was  biased  towards  one  of 
the  groups. 

Summary 

A  convenience  sampling  technique  was  employed  in 
collecting  data  from  configuration  and  data  managers  and 
respective  program  managers.  The  method  of  data  collection 
employed  was  the  survey,  specifically  that  of  the  personal 
interview.  Five  hypotheses,  related  directly  to  research 
questions  posed,  were  tested  for  validity  by  various 
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statistical  analyses.  Subsequent  statistical  analyses  were 
performed  to  indicate  both  the  level  of  measurement 
reliability  of  the  instrument  employed  and  the  degree  of 
convergent  validity  on  key  constructs. 
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IV.  Results  and  Findings 


Introduction 

A  total  of  27  matched-pair  interviews  were  conducted  as  a 
means  of  data  collection  to  satisfy  the  goals  of  this 
research  problem.  The  number  of  interviews  conducted 
constitutes  93%  of  the  total  configuration  and  data 
management  population  in  the  three  organizations  surveyed. 
Subsequent  statistical  analyses  were  performed  as  described 
in  chapter  III. 

This  chapter  present  a  discussion  of  significant  findings 
(both  qualitative  and  quantitative)  related  to  the  five 
research  hypotheses  posed  by  this  project,  as  well  as  other 
findings  relative  to  this  research  effort.  An  assessment  of 
the  validity  and  reliability  of  the  survey  used  for  data 
collection  concludes  this  chapter. 

Tests  of  Hypotheses 

Each  of  the  five  hypotheses  is  discussed  in  the  following 
format:  statement  of  the  hypothesis,  statistical  analysis, 

and  qualitative  inferences. 

Hypothesis  1 .  Specialized  competence  ratings  of 
configuration  and  data  managers  does  not  vary 
significantly  and  are  not  dependent  upon  the  educational 
background  of  the  manager. 
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Statistical  Results.  An  unbalanced  single-factor 
analysis  of  variance  (ANOVA)  (alpha  level  =  0.05)  was  used 
to  determine  significant  differences  between  the  educational 
background  of  configuration  and  data  managers  and  perceived 
competence.  Tables  I  and  II  represent  summaries  of  ANOVA 
testing  for  the  intrinsic  and  extrinsic  ratings, 
respectively.  Appendix  F  provides  the  raw  data  of  the 
configuration  and  data  manager's  educational  categories  and 
competence  ratings  (intrinsic  and  extrinsic)  obtained 
through  the  survey. 
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Table  I.  Bonferroni  Mul t i compa r i son  of  Configuration 
Data  Managers  by  Academic  Degree  (Intrinsic) 

ANALYSIS  OF  VARIANCE  PROCEDURE 
BONFERRONI  (DUNN)  T  TESTS  FOR  VARIABLE:  COMPETENCE 


ACADEMIC 

LOWER 

DIFFERENCE 

UPPER 

EDUCATION 

CONFIDENCE 

BETWEEN 

CONFIDENCE 

COMPARISON 

LIMIT 

MEANS 

LIMIT 

(  CATEGORY) 

(B) 

(D) 

TECHNICAL/ 

ASSOCIATE 

0.024 

3.681 

7 .337 

•k  -k  -k 

(B) 

(E) 

TECHNICAL/ 

HIGH  SCHOOL 

3  .  398 

6 .856 

10.313 

k  k  -k 

(D) 

(  E) 

ASSOCIATE/ 

HlLrM  SCHOOL 

-0 . 395 

3  .175 

6.745 

***  SIGNIFICANT  DIFFERENCE  BETWEEN  MEAN  AT  ALPHA  =  0.05 

ALPHA=0 . 0  5  CONFIDENCE=0 . 9  5  DF  =  24 

NOTE:  THIS  TEST  CONTROLS  THE  TYPE  I  EXFERIMENTWI SE  ERROR 
RATE  BUT  GENERALLY  HAS  A  HIGHER  TYPE  II  ERROR  RATE  THAN 
TUKEY'S.  CRITICAL  VALUE  OF  T=2 . 57364 
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The  Bonferroni  t  test  was  used  to  compare  the  competence 
rating  of  configuration  and  data  managers  within  different 
education  categories.  Prior  to  a  discussion  of  statistical 
analysis,  terms  associated  with  the  Bonferroni  t  test  must 
be  defined. 

In  determining  if  the  differences  between  educational 
categories  are  significant  (ANOVA  analysis),  two  types  of 
errors  may  be  committed.  These  errors,  type  I  and  type  II, 
are  based  on  the  relationship  between  the  hypothesis  and  the 
decision  made  following  statistical  analysis.  In  each  case, 
statistical  analysis  begins  with  the  statement  of  a  null 
hypothesis.  Type  I  errors  occur  when  the  null  hypothesis  is 
rejected,  when  the  proposed  hypothesis  is  true.  For  this 
project,  stating  that  one  education  level  and  the  related 
competence  ratings  differ  significantly  from  the  other 
levels  when  in  fact  there  is  no  significant  difference 
between  levels,  would  constitute  a  type  I  error.  A  type  II 
error  occurs  when  the  null  hypothesis  is  accepted  when  the 
hypothesis  posed  should  be  rejected.  Designating  the  alpha 
level  of  significance  to  be  0.05  decreases  the  possibility 
of  a  type  I  error  occurring.  The  smaller  the  type  I  error 
(alpha),  the  less  likely  it  is  that  a  deviant  sample 
response  will  cause  the  null  hypothesis  to  be  rejected.  In 
testing  the  data,  a  critical  value  (T=2.5734),  or  test 
statistics  value,  is  calculated  from  the  degrees  of  freedom 
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(number  of  samples  taken  minus  the  number  of  different 
categories  or  24)  and  the  designated  alpha  level  (0.05).  If 
the  comparisons  of  variances  of  the  two  categories  is 
greater  than  the  critical  "T  value",  the  resultant 
associated  probability  will  be  less  than  the  alpha  level 
(0.05),  and  it  can  be  concluded  that  the  two  categories 
being  compared  are  significantly  different.  By  setting  a 
range  or,  statistically  speaking,  a  confidence  interval  (or 
limit)  of  95%,  conclusions  can  be  inferred  regarding  the 
relationship  of  the  variable  with  great  accuracy.  In  Table 
I,  the  upper  and  lower  confidence  limits  (reflecting  the 
highest  and  lowest  expected  competence  rating  for  an 
interviewee  with  a  certain  education)  bound  the  range  of 
numerical  values  into  which  the  difference  between  the 
competence  ratings  of  two  individuals  with  the  noted 
educational  backgrounds  would  be  expected  to  fall. 

Information  describing  the  relationship  between 
educational  background  and  perceived  competence  levels  is 
shown  in  table  I.  Categories  A  (beyond  technical  degree) 
and  C  ( non-technical  degree)  were  not  included  in  the 
statistical  .  I  of  t h ■>  c  pi  ,  j  e  ~  since  none  of  the 
configuration  and  data  managers  interviewed  possessed  these 
educational  backgrounds.  The  self-professed  (intrinsic) 
educational  category  specialized  competence  rating  means 
were:  Technical  Degree  31.56,  Associates  Degree  27.87,  and 
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High  School  Diploma  24.70.  Analysis  of  data  indicated  that 
configuration  and  data  personnel  within  category  B 
(technical  degree)  possessed  the  highest  self-rated 
competencies  (as  shown  by  the  greatest  difference  in  means) 
relative  to  the  other  educational  categories.  This  is 
reflected  in  the  analysis  of  the  variance  in  Table  I.  For 
example,  when  comparing  configuration  and  data  managers 
possessing  a  technical  degree  to  those  possessing  a  high 
school  diploma,  the  manager  possessing  a  technical  degree 
can  be  expected  to  exhibit  a  competence  rating  ranging  from 
3.398  to  10.313  points  higher  (with  6.586  points  being  the 
average)  than  the  manager  possessing  a  high  school  diploma. 

When  the  technical  degree  (B)  was  compared  to  the 
associates  degree  (D)  and  subsequently  to  the  high  school 
diploma  (E),  the  resulting  probabilities  that  the  competence 
would  be  the  same  for  both  between  the  educational 
categories  were  0.0097  and  0.0001,  respectively.  Both 
calculated  probabilities  fall  below  the  designated  0.05 
alpha  level,  making  the  groups  (B:D,  B : E )  significantly 
different.  A  statistically  significant  difference  did  not 
exist  however  between  the  associates  degree  (D)  and  the  high 
school  diploma  (E),  where  the  probability  was  calculated  to 
be  0.0523.  A  calculated  probability  greater  than  the  alpha 
level  indicates  that  no  difference  in  competence  exists 
between  the  educational  categories  of  associates  degree  and 
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high  school  diploma.  However,  since  the  probability  was 
very  close  to  being  statistically  significant,  it  can  be 
inferred  that  additional  education  appeared  to  have  some 
influence  upon  intrinsic  competence  ratings. 

Results  of  data  analysis  presented  in  Table  II  extrinsic 
(prog, am  managers  perceived)  competence  ratings  indicate  a 
similar  trend  in  the  education/competency  relationship  to 
those  obtained  for  the  intrinsic  ( conf iguration  and  data 
manager's  self  perceived)  competence  ratings.  The 
educational  category  specialized  competence  (extrinsic) 
means  were:  Technical  Degree  31.22,  Associates  Degree 
27.75,  and  High  School  Diploma  23.10.  Differences  in 
extrinsic  competency  ratings  between  configuration  and  data 
functionals  possessing  a  technical  degree  (B)  and  those 
possessing  either  an  associate  degree  (D)  or  a  high  school 
diploma  (E)  were  statistically  significant  (probabilities 
were  0.0083  and  0.0001,  respectively).  Similar  to  data 
analysis  for  self-rated  comp?tency,  configuration  and  data 
personnel  within  category  B  (technical  degree),  possessed 
the  highest  extrinsically  rated  competencies  relative  to  the 
other  educational  backgrounds. 

In  contrast  to  the  intrinsic  competency  results,  where  no 
statistical  difference  was  observed  between  categories  D  and 
E  (associates  degree  versus  high  school  diploma/,  the 
extrinsic  competency  ratings  did  yield  a  statistically 
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significant  difference  between  categories  (probability  value 
=  0.0048).  The  program  manager's  ratings  definitely  equated 
that  additional  educational  attainment  with  competence. 
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Table  II.  Bonferroni  Mul t i compa r i son  of  Configuration 
Data  Managers  by  Academic  Degree  (Extrinsic) 


ANALYSIS  OF  VARIANCE  PROCEDURE 


BONFERRONI  (DUNN) 

T  TESTS  FOR 

VARIABLE: 

COMPETENCE 

ACADEMIC 

LOWER 

DIFFERENCE 

UPPER 

EDUCATION 

CONFIDENCE 

BETWEEN 

CONFIDENCE 

COMPARISON 

LIMIT 

MEANS 

LIMIT 

( CATEGORY) 

(B)  TECHNICAL/ 


(D) 

ASSOCIATE 

0.346 

3 . 472 

6 . 598 

■k  k  k 

(B) 

TECHNICAL/ 

(E) 

HIGH  SCHOOL 

5  . 166 

8 .122 

11 . 078 

*  *  * 

(D) 

ASSOCIATE/ 

(E) 

HIGH  SCHOOL 

1 .  598 

4.650 

7.702 

■k  k  k 

***  SIGNIFICANT  DIFFERENCE  BETWEEN  MEAN  AT  ALPHA  =  0.05 

ALPHA=0 . 0  5  CONFIDENCE=0 . 95  DF=24 

NOTE:  THIS  TEST  CONTROLS  THE  TYPE  I  EXPERIMENTWI SE  ERROR 
RATE  BUT  GENERALLY  HAS  A  HIGHER  TYPE  II  ERROR  RATE  THAN 
TUKEY'S  CRITICAL  VALUE  OF  T=2. 57364 
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Qualitative  Inferences.  Figure  9  presents  a  scatter  plot 


of  competency  versus  educational  level.  This  plot  was 
constructed  from  the  configuration  and  data  managers  self- 
rated  competence  and  program  managers  extrinsic  rated 
competence  data.  It  should  be  understood  that  although  the 
results  obtained  do  not  necessarily  attribute  a  high 
absolute  degree  of  actual  competency  to  a  technical  degree 
(educational  category  B),  the  results  do  attribute  a  higher 
relative  degree  of  perceived  competency  to  the  technical 
degree.  Even  though  this  research  project  did  not 
quantitatively  determine  specifically  how  possessing  a 
technical  degree  affects  job  performance,  it  should  be  noted 
that  the  perception  of  competence  of  a  configuration  and 
data  manager  possessing  a  certain  education  is  often  more 
important  than  actual  job  performance.  In  general,  the 
positive  perception  generated  by  the  technical  degree  may 
result  in  the  program  manager  having  more  confidence  in,  and 
perhaps  utilizing  to  a  greater  extent,  a  conf i gu ra t i on  and 
data  manager  who  possesses  a  technical  degree  than  a  manager 
who  does  not. 
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Figure  9  .  Competence  Ratings  Versus  Education  Level 
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Hypothesis  2.  Extrinsic  and  intrinsic  performance 
ratings  of  a  configuration  and  data  managers  specialized 
competence  are  positively  correlated  with  respective 
ratings  of  aptitude. 

Statistical  Results.  Correlation  analyses  between 
ratings  of  specialized  competence  and  aptitude  (ability  to 
pursue  unclear  issues  and  assimilate  new  information)  were 
performed  using  the  Pearson  r  correlation  coefficient.  The 
first  analysis  examined  the  relationship  between  intrinsic 
aptitude  and  specialized  competence  of  conf i gu r at i on  and 
data  managers.  The  second  analysis  examined  the 
relationship  between  the  extrinsic  (program  manager 
perceived)  aptitude  ratings  of  the  configuration  and  data 
managers  and  extrinsic  related  specialized  competence.  For 
the  intrinsic  (self)  ratings  of  aptitude  versus  specialized 
competence,  correlation  coefficients  were  of  the  magnitude 
of  0.8803.  Similarly,  for  extrinsic  ratings,  the  correlated 
relationship  yielded  a  coefficient  of  0.7953.  Since  0.6  is 
considered  good  correlation  and  0.8  is  excellent,  these 
values  indicate  that  a  high  positive  correlation  between 
both  intrinsic  and  extrinsic  data  sets  exists. 

Qualitative  Inferences.  A  high  positive  correlation 
(for  both  intrinsic  and  extrinsic  ratings)  between  the  level 
of  a  configuration  and  data  manager's  competence  and  the 
level  of  aptitude  indicates  that  aptitude  is  a  very  strong 
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predictor  of  competence.  This  strong  positive  correlation 
suggests  that  labeling  an  individual  as  competent  or  not 
competent,  based  solely  upon  the  individual's  academic 
degree,  may  not  be  appropriate.  Rather,  the  individual's 
aptitude  or  ability  to  pursue  unclear  issues  and  assimilate 
new  information  would  be  at  least  equally  as  important  as 
education  in  assessing  the  individual's  level  of  competence. 
Hypothesis  3.  The  perceived  importance  of  proficient, 
specialized  competence  ratings  (extrinsic  and  intrinsic) 
of  a  configuration  and  data  manager  differs  s igni f i can t ly 
with  the  system  program  phase. 

Statistical  Results.  This  analysis  was  intended  to 
determine  if  a  significant  difference  exists  among  the 
ratings  of  the  importance  of  specialized  configuration  and 
data  management  competence  for  various  program  phases. 
However,  the  statistical  analyses  were  not  performed  since 
most  of  the  configuration  and  data  management  acquisition 
programs  were  in  one  (full  scale  development)  of  the  five 
phases.  Since  all  five  program  phases  were  not  represented, 
results  of  statistical  analysis  would  be  inconclusive  due  to 
the  small  sample  size  cor  the  other  program  phases. 
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Qualitative  Inferences.  While  interviewing 
configuration  and  data  managers  and  respective  program 
managers,  both  stated  that  conf i gu ra t i on  and  data  management 
issues  were  of  greatest  importance  during  the  full  scale 
development  phase  and  the  production  phase.  This  perception 
supports  the  research  findings  of  Thamhain,  Miller  and 
Baumgarter,  who  all  found  that  the  intensity  of  management 
conflicts  associated  with  programmatic  issues  varies  with 
the  acquisition  program  phase  and  is  most  prevalent  in  full 
scale  development  and  production  (3,24,31). 

Hypothesis  4.  A  positive  correlation  exists  between 
the  overall  performance  rating  of  a  configuration  and 
data  manager's  specialized  competence  and  the  program 
manager's  utilization  of  the  specialized  competence  for 
accomplishing  program  objectives. 

Statistical  Results.  Correlation  analyses  between 
specialized  competence  and  utilization  of  specialized 
competence  oy  a  program  manager  were  performed  using  the 
Pearson  r  correlation  coefficient.  The  correlation  analyses 
were  performed  twice;  once  using  intrinsic  ( sel f-pe r ce i ved ) 
ratings  of  specialized  competence  and  utilization  by 
respective  program  managers,  and  once  using  the  extrinsic 
(program  manager  perceived)  ratings  of  specialized 
competence  and  utilization  by  the  program  manager.  The 
utilization  of  the  configuration  and  data  managers,  as 
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related  to  intrinsic  ratings  of  specialized  competence 
yielded  a  correlation  rating  of  -0.1322.  A  correlation 
value  of  essentially  zero  indicates  that  configuration  and 
data  managers  believe  that  utilization  of  their  specialized 
competence  by  a  program  manager  is  not  related  to  self-rated 
competence.  The  program  manager's  (extrinsic)  ratings  of 
utilization  as  related  to  competence  produced  a  correlation 
relationship  of  0.4452.  This  correlation  rating  is  marginal 
(0.6  is  considered  good)  and  indicates  that  utilization  of 
specialized  competence  by  the  program  manager  is  probably 
unrelated  to  specialized  competence. 

Qualitative  Inferences.  During  the  survey  and 
interview  process,  program  managers  rated  many  of  the 
configuration  and  data  managers  as  possessing  a  certain 
degree  (many  with  a  high  degree)  of  specialized  competence. 
However,  the  marginal  correlation  rating  (0.4452)  indicates 
that  a  program  manager's  utilization  of  the  configuration 
and  data  manager  is  not  related  to  that  perceived 
competence ,  even  for  the  we 11- qualified  configuration  and 
data  managers.  In  other  words,  greater  competence  does  not 
mean  greater  utilization  by  the  program  manager.  The 
competence  rating  of  -0.1322  for  configuration  and  data 
managers  appears  to  confirm  this.  This  is  probably  the  most 
important  finding  of  this  project. 
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Discussions  with  configuration  and  data  managers  revealed 
a  prevailing  opinion  that  a  configuration  and  data  manager 
is  utilized  only  in  a  "fire-fighting"  mode.  This  crisis 
mode  requires  that  the  program  manager  utilize  his 
functional  personnel  in  order  to  attain  instantaneous 
results  to  meet  critical  program  deadlines.  Outside  of  the 
crisis  mode,  however,  conf iguration  and  data  functionals  are 
utilized  at  a  minimum  level.  In  interviews  conducted  with 
program  managers  concerning  the  utilization  issue  (and 
especially  with  managers  who  did  not  utilize  configuration 
and  data  functionals),  a  majority  stated  that  they  were 
better  able  to  perform  the  configuration  and  data  manager's 
duties  than  was  the  conf iguration  and  data  manager  himself. 
The  program  managers  partially  justified  this  by  saying  that 
the  time  required  to  relay  the  tasks  to  the  configuration 
and  data  manager  was  greater  than  the  time  required  to 
accomplish  the  task.  One  program  manager  went  so  far  as  to 
create  a  self-sufficient  data  tracking  system  for  the 
purpose  of  monitoring  the  contractor's  data  submissions 
independent  of  his  configuration  and  data  functional. 
Hypothesis  5.  The  overall  performance  rating  of  a 
conf igurat. m  and  data  manager's  competence  does  not  vary 
among  ASD  organizations.  Such  variation  may  be 
attributed  to  the  internal  polices  of  a  given 
organi zation . 
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Statistical  Results.  An  unbalanced  single-factor 
analysis  of  variance  (ANOVA)  (alpha  level  =  0.05)  was 
utilized  in  determining  if  significant  differences  exist 
between  of  the  three  sample  group  organizations. 

Performance  of  the  ANOVA  test  between  the  three 
organizations  indicated  that  no  statistically  significant 
difference  in  intrinsic  or  extrinsic  competence  rating 
existed  between  organizations.  The  organization  competence 
means  were  intrinsically  AE-28.62,  RW-27.73,  and  YW-27.50 
and  extrinsically  AE-27.25,  rw-27.45,  and  YW-26.75. 
Supporting  analysis  results  are  presented  in  Table  III. 
Comparison  probabilities  between  organizational  groupings 
were  all  calculated  to  be  greater  than  the  designated  0.05 
alpha  level. 

Qualitative  Inferences.  The  internal  policies  and 
operating  procedures  of  the  configuration  and  data 
management  area  have  been  detailed  in  the  appropriate  OoD 
and  Air  Force  Regulations  and  military  standards.  The  minor 
variations  in  internal  organizational  policies  and 
procedures,  for  the  or gan i za t i ons  surveyed  in  this  research 
project,  did  not  have  an  apparent  influence  on  the  level  of 
intrinsically  or  extrinsically  perceived  specialized 
competence  of  configuration  and  data  managers.  While 
policies  among  the  three  organizations  are  different,  the 
differences  exist  primarily  in  the  detailed  execution  of  key 
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configuration  and  data  management  tasks.  During 
questioning,  however,  configuration  and  data  managers  stated 
that  actual  task  performance  was  frequently  not  in 
accordance  with  organizational  policies  due  to  of  the 
program  manager's  management  style.  Such  deviation  from 
organizational  policies  for  various  programs  serves  to 
invalidate  organizational  formalization,  thereby  increasing 
role  ambiguity,  and,  for  the  configuration  and  data  manager, 
increasing  the  difficulty  of  attaining  specialized 
competency . 
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Table  III.  Bonferroni  Multicomparison  of  Configuration 
Data  Managers  by  Organization 


ANALYSIS  OF  VARIANCE  PROCEDURE 
BONFERRONI  (DUNN)  T  TESTS  FOR  VARIABLE:  COMPETENCE 
INTRINSIC  RATINGS 


ORGANIZATIONAL 

COMPARISON 

LOWER 

CONFIDENCE 

LIMIT 

DIFFERENCE 

BETWEEN 

MEANS 

UPPER 

CONFIDENCE 

LIMIT 

AE 

-  RW 

-4 .117 

0 .898 

5.913 

RW 

-  YW 

-4 .788 

0 . 227 

5 .242 

YW 

-  AE 

-6 . 521 

-1 .125 

4  .  271 

ALPHA=0 . 

05  CONFIDENCE* 

0.95  DF=2  4 

EXTRINSIC  RATINGS 

AE 

-  RW 

-5.531 

-0 .204 

5 .122 

RW 

-  YW 

-5 . 372 

0 .204 

5  .531 

YW 

-  AE 

-5.482 

-0 .250 

5 . 982 

ALPHA=0 . 

05  CONFIDENCE* 

0.95  DF  =  2  4 

***  SIGNIFICANT  DIFFERENCE  BETWEEN  MEAN  AT  ALPHA  =  0.05 

NOTE:  THIS  TEST  CONTROLS  THE  TYPE  I  EXPERIMENT WISE  ERROR 
RATE  BUT  GENERALLY  HAS  A  HIGHER  TYPE  II  ERROR  RATE  THAN 
TUKEY'S.  CRITICAL  VALUE  OF  T=2. 57364 
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Other  Qualitative  Findings 


During  the  interviews,  configuration  and  data  managers 
emphasized  the  configuration  and  data  management  staff 
agency's  ( ASD/ENO-AWZ )  inability  to  assert  authority  in  both 
the  administration  and  enforcement  of  practices  and 
policies.  Under  the  existing  organizational  structure,  the 
configuration  and  data  management  office  is  subordinate  to 
the  functional  Deputy  for  Engineering.  All  policies  and 
practices  to  be  implemented  by  the  conf iguration  and  data 
management  staff  agency  must  be  coordinated  through,  and 
approved  by,  the  engineering  deputy.  The  proposal  for 
implementation  of  a  particular  policy  to  enhance 
configuration  and  data  management  by  the  staff  agency  may 
not  be  approved  by  the  Deputy  for  Engineering  if  a  conflict 
with  engineering  interest  is  foreseen.  The  intended  purpose 
of  the  configuration  and  data  management  staff  agency 
includes  assuring  the  promulgation  of  good  configuration  and 
data  management  policies,  monitoring  program  office 
implementation  of  configuration  and  data  management 
policies,  and  advising  and  training  program  personnel  in 
"correct"  configuration  and  data  management  practices.  The 
multipurpose  nature  of  the  staff  agency  coupled  with  the 
requirement  to  coordinate  all  activities  through  the 
functional  Deputy  for  Engineering  results  in  staff  agency 
delays  and  (sometimes)  less  than  optimal  revisions  to 
proposed  actions. 
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During  the  interview  process,  questions  concerning  the 
delegation  of  duties  associated  with  the  configuration  and 
data  management  area  were  asked  of  both  configuration  and 
data  managers  and  program  managers.  The  numeric  summation 
of  responses  is  located  in  Appendix  G.  A  majority  of  the 
configuration  and  data  managers  currently  perform  their 
assigned  duties  as  required  by  policy  for  the  normal  order 
of  business  of  their  programs.  However,  as  previously 
mentioned,  utilization  of  the  configuration  and  data  manager 
over  and  above  these  assigned  duties  (other  than  in  special 
program  situations)  is  minimal.  The  configuration  and  data 
managers  and  the  program  managers  did  agree  on  the  duties 
which  should  be  performed  by  the  program  manager  and  those 
which  should  be  performed  by  the  conf iguration  and  data 
manager.  This  breakout  of  responsibilities  differs 
significantly  from  current  practices  and  should  be  employed 
in  the  redefinition  of  the  standard  roles  and 
responsibilities  of  the  program  manager  and  of  the 
configuration  and  data  manager. 

when  asked  to  identify  the  attributes  and  qualities 
possessed  by  a  "proficient"  configuration  and  data  manager, 
no  consensus  was  reached  regarding  the  educational  level 
among  the  configuration  and  data  managers.  Consensus  was 
achieved  in  the  areas  of:  experience  (number  of  years), 
experience  (in  certain  functional  areas)  and  specialized 
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education  (see  Appendix  G).  The  configuration  and  data 
managers  (intrinsic)  stated  that  a  proficient  manager  (for 
multiple  small  programs  4n  a  matrixed  situation)  must  have 
at  least  two  years  of  configuration  and  data  management 
experience.  Further,  a  proficient  manager  is  very  familiar 
with  the  functional  practices  and  concepts  associated  with 
engineering  and  logistics.  With  the  goal  of  increasing 
productivity,  the  configuration  and  data  managers  expressed 
strong  interest  in  the  following  educational  areas: 
software  documentation,  engineering  drawings  regulations, 
system  engineering  application,  and  automated  databases. 

From  the  extrinsic  viewpoint,  desired  attributes  of  a 
configuration  and  data  manager  were  easily  identified. 
Ideally,  program  managers  very  strongly  desired  a 
configuration  and  data  manager  possessing  a  technical  degree 
with  an  understanding  of  the  engineering  functional  area. 
Believing  that  increased  productivity  is  achieved  through 
education  and  supplemental  training  programs,  the  program 
managers  duplicated  the  configuration  and  data  manager's 
response  with  regard  to  educational  interest  areas. 

Traininq/Education .  An  issue  raised  by  a  majority  of  the 
conf i gu r a t i on  and  data  managers  interviewed  was  the  subject 
of  training  as  a  means  for  enhanced  job  performance. 

Relevant  training,  timely  training  and  on-going  education 
were  the  specific  concerns  voiced  by  configuration  and  data 
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managers.  All  believed  that  their  training  and  education 
were  neglected  to  increase  availability  for  work  and  that 
participation  in  training  and  education  programs  would  serve 
to  increase  their  effectiveness  and  their  overall  competence 
in  the  performance  of  their  duties. 

Relevant  training  coursework,  designed  to  create  an 
awareness  and  understanding  of  configuration  and  data 
management  issues  directly  related  to  the  duties  of  the 
functional,  are  offered  by  the  Air  Force  Institute  of 
Technology.  Though  courses  are  available  in  conf i gu ra t i on , 
data,  and  engineering  data  management,  conf iguration  and 
data  functionals  concur  that  detail  issues  and  subject 
material  deemed  important  are  not  adequately  addressed  by 
the  AFIT  courses.  Rather,  a  superficial  ’’theory-oriented” 
approach  is  taken.  Specifically,  several  configuration  and 
data  managers  stated  that  the  AFIT  engineering  data 
management  course  was  a  "complete  waste  of  time  and  money" 
at  the  time  they  attended  it;  the  instructor  had  no  greater 
knowledge  in  the  engineering  drawing  area  than  some  of  the 
attending  students.  And  if  AFIT  is  only  providing  the 
background,  the  ASD  staff  agency  and  program  office  On-The- 
Job  training  are  not  providing  che  additional  details 
required  for  the  configuration  and  data  managers  to 
adequately  learn  their  assigned  duties. 
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Timely  consultations  by  directing  specific  questions  to 


an  expert  in  the  configuration  and  data  management  area 
about  situations  as  they  arise  is  also  an  important  concern. 
Ideally,  within  ASD,  configuration  and  data  management 
personnel  receive  official  guidance  from  the  staff  agency 
( ASD/AWZ ) .  However,  a  majority  of  the  configuration  and 
data  personnel  believe  that  staff  agency  guidance  or  support 
is  not  readily  available.  Needs  of,  and  requests  by,  the 
program  office  configuration  and  data  managers  are  net  being 
adequately  handled  by  the  staff  agency. 

Continuing,  or  on-going,  education,  available  through 
national  and  regional  seminars  and  conferences  would  also  be 
useful  in  enhancing  the  capabilities  of  the  configuration 
and  data  manager.  The  configuration  and  data  managers 
believe  that  a  greater  opportunity  to  attend  and  participate 
in  configuration  and  data  seminars  would  increase  their 
productivity . 

Significant  Findings 

During  the  data  collection  process,  discussions  with 

conf iguration  and  data  managers  for  this  project  brought  out 

several  significant  items.  These  findings  are: 

(1)  The  con f i gu ra t i on  and  data  manager  is  relatively 
insignificant  when  compared  to  other  functional 
program  personnel. 


Interview  responses  by  program  managers  and  configuration 
and  data  managers  (extrinsic  and  intrinsic,  respectively)  to 
questions  3.7  and  1.7  of  their  surveys  and  subsequent 
analyses  of  the  responses,  indicate  a  prevailing  attitude 
that  program  managers  consider  the  configuration  and  data 
manager  "non-essential"  to  program  success  but  that  they 
consider  the  configuration  and  data  management  duties 
essential . 

(2)  In  general,  there  are  few  sufficiently  knowledgeable 
configuration  and  data  management  personnel. 

Further  complicating  the  image  problem  of  the 
conf iruration  and  data  manager  is  the  fact  that  the 
configuration  and  data  management  workforce  surveyed  is 
currently  comprised  of  a  significant  percentage  (60%)  of 
trainees,  with  few  well-qualified  individuals  among  tne 
ranks.  Trainees  yet  unfamiliar  with  many  aspects  of 
configuration  and  data  management  are  assigned  to 
acquisition  programs  as  the  only  configuration  and  data 
management  resource.  Clearly,  such  trainees  have  great 
difficulty  operating  effectively  under  these  circumstances. 
As  stated  previously,  configuration  and  data  managers  felt 
that  at  least  two  years  of  experience  was  required  to 
provide  adequate  expertise  for  these  matrix  positions. 
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(3)  In  general,  the  program  manager  possesses  an 
inadequate  understanding  of  the  breadth  of  the 
configuration  and  data  management  and  of  the  duties 
performed  by  a  configuration  and  data  manager. 

The  program  manager  does  not  help  the  configuration  and 
data  manager  to  gain  the  needed  experience  Many  program 
managers  have  an  incomplete  understanding  of  configuration 
and  data  management  issues,  or  of  the  functions  the 
configuration  and  data  manager  should  accomplish,  and  so  do 
not  know  when  or  how  to  utilize  the  configuration  and  data 
manager.  Ultimately,  the  outcome  is  under-utilization  of 
the  conf iguration  and  data  manager  and  a  lack  of  training 
situations  to  increase  their  competence. 

(4)  Utilization  of  a  configuration  and  data  manager  is 
unrelated  to  the  individual's  competency  level. 

As  noted  in  chaoter  two  of  this  thesis  (regarding  the 
importance  of  competence),  the  program  manager's  lack  of 
confidence  in  the  abilities  of  the  configuration  and  data 
manager,  coupled  with  the  physical  non-availability  of  the 
configuration  and  data  manager,  may  result  in  non¬ 
utilization  of  this  functional.  Configuration  and  data 
information  and  activities  deemed  necessary  for  program  goal 
accompl i shment  were  often  processed  and  accomplished 
independent  of  the  configuration  and  data  functional.  Far 
too  often,  the  rationale  used  to  make  the  utilisation  versus 


non-utilization  decision  was  based  upon  the  program 
manager's  own  ability  to  comprehend  configuration  and  data 
management  issues,  his  opinion  of  the  time  required  for  the 
task  completion  and  of  the  configuration  and  data  manager's 
familiarity  with  program  issues,  and  the  ready  availability 
of  the  configuration  and  data  manager.  Since  most  program 
managers  opt  to  bypass  the  configuration  and  data  managers, 
this  can  undermine  the  effectiveness  of  the  entire 
acquisition  process.  Since  the  program  manager  is 
ultimately  responsible  for  development  of  the  new 
acquisition  system,  decisions  pertaining  to  utilization  or 
non-utilization  of  functional  personnel  are  made  at  his 
discretion.  Interview  responses,  from  both  program  managers 
and  configuration  and  data  managers  indicate  that 
configuration  and  data  managers  are  utilized  far  less 
frequently  (other  than  in  a  crisis  situation)  than  other 
functional  program  personnel.  The  analysis  of  the  interview 
responses  indicates  that  individual  competencies  of 
configuration  and  data  managers,  whether  highly  competent  or 
trainee  in  nature,  do  not  significantly  influence  the 
utilization  or  non-utilization  of  the  configuration  and  data 
manager  by  a  program  mmager. 
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Reliability  and  Convergent  Validity 


Measurement  Reliability.  Cronbach  alpha  reliability 
coefficients  were  calculated  for  the  configuration  and  data 
manager  survey  instrument  and  for  the  program  manager  survey 
instrument.  The  reliability  measured  the  accuracy,  on  the 
average,  of  the  scores  for  the  composite  (summed  and 
averaged)  variables.  Reliability  may  range  from  0.00  (all 
error  is  attributable  to  measurement),  to  1.00  (no  error  is 
due  to  measurement).  The  reliability  coefficient  for  the 
five  item  specialized  competence  index  was  0.75.  A  0.82 
Cronbach  alpha  was  calculated  for  the  two-item  aptitude 
index.  The  two-item  specialized  importance  index  exhibited 
a  0.88  Cronbach  alpha.  Therefore,  the  composite  variables 
can  be  deemed  very  reliable  predictors  in  their  areas. 

Convergent  Validity.  Paired  t-values  were  calculated  to 
assess  how  closely  the  opinions  of  the  configuration  and 
data  manager  and  his/her  respective  program  manager  matched. 
The  paired  t-test  produced  no  significant  differences 
between  the  intrinsic  and  extrinsic  ratings  in  specialized 
competence,  specialized  importance,  and  utilization  of 
configuration  and  data  manager  competence.  These  results 
indicate  that  the  program  managers  (extrinsic)  ratings  in 
these  areas  were  approximately  equivalent  to  the 
conf i gu r a t i on  and  data  manager  intrinsic  (self)  ratings. 
However,  a  significant  difference  was  noted  in  the  ratings 
of  specialized  altitude,  in  that  configuration  and  data 
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managers  rated  their  own  aptitude  higher  than  the  extrinsic 
ratings  generated  by  program  managers.  The  difference 
between  aptitude  ratings  is  not  unexpected  since  in  general, 
an  external  observer  is  better  qualified  to  judge  an 
individual's  inquisitiveness  rather  than  the  individual 
evaluating  his  own  inquisitiveness.  Therefore  the 
questionnaire  overall  was  not  biased  between  groups. 
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V. 


Conclusions  and  Recommendations 


Introduction 

The  objective  of  this  research  project  was  to  identify 
potential  solutions  to  the  problem  of  limited  productivity 
within  the  matrix-structured  environment,  and  subsequently, 
to  recommend  strategies  for  enhanced  organizational 
productivity  in  the  configuration  and  data  management  area. 
This  chapter  contains  strategies  for  productivity 
enhancements  and  recommendations  for  heightened  matrix 
organizational  productivity  based  on  the  significant 
findings  of  this  project.  Suggestions  for  future  pertinent 
research  are  also  proposed. 

Assumptions  and  Limitations 

Currently,  a  data  collection  device  is  not  available 
which  provides  a  definitive  determination  of  an  individual's 
true  level  of  competence.  Recognizing  the  absence  of  some 
essential,  yet  unmeasurable,  indicators  of  competence,  the 
survey  instrument  employed  was  determined  to  be  the  most 
suitable  means  available  based  on  the  current  published 
literature.  The  data  collection  device  for  this  project, 
and  the  modifications  (detail  information)  made  to  the 
device  to  tailor  it  for  this  project,  were  validated  by 
configuration  and  data  management  experts.  Survey  data  was 
subjected  to  statistical  analysis  to  assess  validity.  The 
results  of  the  validity  assessment  are  presented  in  chapter 
four. 
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Conclus i o ns 


The  goal  of  this  research  was  to  determine  factors,  if 
any,  which  may  contribute  to  decreased  productivity  within 
the  matrixed  configuration  and  data  management  functional 
office.  Among  factors  identified  by  this  project  which 
contribute  to  decreased  productivity,  the  non-utilization  of 
confiqu ration  and  data  managers  by  respective  program 
managers,  regardless  of  competence,  and  the  physical  non¬ 
availability  of  configuration  and  data  managers  to  those 
program  managers  are  the  two  greatest  concerns .  Correction 
can  only  be  accomplished  by  changing  the  competence 
perceptions  of  the  program  manager  and  by  assuring  the 
program  manager  that  the  configuration  and  data  manager  is 
familiar  with  program  issues  and  readily  available.  Efforts 
to  improve  the  competence  of  the  configuration  and  data 
managers  through  the  implementation  of  plans  to  enhance 
their  educational  level,  aptitude  or  specialized  competence 
will  be  ineffective  unless  they  are  complemented  by 
organizational  changes  which  make  the  configuration  and  data 
manager  immediately  available  to  the  program  manager  when 
needed.  The  enhancements,  by  themselves,  will  not  guarantee 
that  the  configuration  and  data  functional  will  be  utilized 
more  and  hence,  that  their  productivity  will  improve. 
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While  these  improvements  in  availability  and  utilization 
may  be  difficult  to  achieve,  there  are  other  factors 
identified  by  this  project  which  contribute  to  decreased 
productivity  and  which  are  more  easily  corrected.  Two 
significant  problems  are  the  abundance  of  configuration  and 
data  management  trainees  and  the  non-standard  configuration 
and  data  management  practices  being  used  in  the 
organizations.  However,  while  action  should  be  taken  to 
assign  more  experienced  personnel  to  these  matrix 
organizations  and  to  standardize  the  responsibilities  and 
practices,  unless  the  utilization/availability  situation  is 
corrected,  productivity  improvements  are  likely  to  be  small. 
Only  when  the  configuration  and  data  manager  is  readily 
available  to  a  program  manager  who  recognizes  the 
"essentiality"  of  the  role  and  contributions  of 
configuration  and  data  managers  can  improvements  in  other 
"competence"  factors  be  implemented  with  expectation  for 
significantly  enhanced  productivity. 

Productivity  Enhancement  Strategies 

Regardless  of  which  of  the  following  recommendations  are 
chosen,  the  following  four  strategies  for  productivity 
enhancement  must  be  implemented. 

First,  as  stated  in  chapter  four  of  this  thesis,  program 
managers  recognize  the  importance  of  con f i gu r a t i on  and  data 
management,  but  they  are  generally  unfamiliar  with  the 
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totality  of  the  configuration  and  data  manager's 
responsibilities.  In  order  to  correct  this  deficiency,  the 
exposure  of  all  program  management  personnel  to 
configuration  and  data  management  practices  must  be 
increased  by  offering  conf iguration  and  data  management 
coursework  to  all  program  management  personnel.  The  program 
managers  must  be  convinced  of  the  importance  of  involving 
the  configuration  and  data  manager  in  program  activities. 
Exposure  to  configuration  and  data  management  coursework 
would  apprise  the  program  manager  about  the  number  and  scope 
of  the  key  configuration  and  data  management  roles  and 
responsibilities  (such  as  the  data  requirements  review 
board,  configuration  audits,  and  engineering  data 
management)  and  would  increase  the  likelihood  that  they 
would  utilize  the  configuration  and  data  management 
functional . 

Second,  the  findings  of  chapter  four  of  this  thesis 
discussed  the  fact  that  the  configuration  and  data 
management  staff  office  is  subordinate  to  the  functional 
deputy  of  engineering.  For  optimal  functioning  of  the 
acquisition  process,  the  staff  agency  must  be  able  to 
independently  direct  and  enforce  policies  concerning  issues 
relating  to  configuration  and  data  management. 

Specifically,  the  configuration  and  data  management  staff 
agency  must  be  removed  from  the  command  of  the  functional 
deputy  of  engineering  and  given  equal  organizational  status. 
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Third,  educating  "trainees"  to  the  discipline  of 
configuration  and  data  management,  as  well  as  providing 
updating  education  (seminars)  for  qualified  configuration 
and  data  managers,  is  essential.  Insufficient  training  was 
identified  as  the  major  deficiency  within  the  conf iguration 
and  data  management  discipline.  The  Professional  Continuing 
Education  courses  offered  through  AFIT  are  intended  to 
provide  trainees  with  a  foundation  on  the  general  issues 
relating  to  configuration  and  data  management. 

Configuration  and  data  managers  interviewed  for  this  project 
did  not  believe  that  the  engineering  data  and  data 
management  courses  were  achieving  the  intended  objective. 
Further,  advanced  courses  or  training  providing  timely 
guidance  on  system  program  office  problems  are  generally 
unavailable.  A  comprehensive  education  and  training  program 
in  the  work  environment  must  be  established  and  implemented 
to  supplement  the  AFIT  courses.  The  training  program, 
generated  and  maintained  through  the  staff  agency,  should  be 
administrated  by  each  system  program  office.  Such  training 
would  ensure  that  configuration  and  data  trainees  are 
brought  "up  to  speed"  in  a  thorough  and  prompt  manner. 
Periodic  updating  for  experienced  configuration  and  data 
personnel  must  also  be  accomplished  through  "refresher" 
courses  or  seminars,  sponsored  by  the  ASD  configuration  and 
data  management  staff  agency.  Refresher  curricula  must  be 
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specifically  directed  to  the  new  or  changing  aspects  of 
configuration  and  data  management. 

Fourth,  a  standardized  set  of  configuration  and  data 
management  practices  and  procedures  must  be  developed  and 
enacted.  Deviations  from  the  standardized  protocol  will  be 
allowed,  with  approval  from  the  staff  agency,  to  accommodate 
individual  program  uniqueness.  Within  and  among  each  of  the 
three  organizations  surveyed,  a  lack  of  standardization  of 
configuration  and  data  management  procedures  was  clearly 
evident.  Coramuni ca t i on  between  configuration  and  data 
management  organizations  must  be  established  and  maintained, 
thereby  ensuring  that  newly  developed  policies  and 
procedures  are  implemented  and  followed  and  that  new 
innovations  are  quickly  shared  with  other  conf i gu ra t i on  and 
data  managers. 

Recommendations 

This  research  project  proposes  two  alternative 
recommendations  for  enhanced  matrix  organizational 
productivity  with  respect  to  the  configuration  and  data 
management  area.  Ultimately,  enhanced  productivity  will  be 
achieved  by  making  changes  to  the  organizational  environment 
and  the  competence  of  the  configuration  and  data  management 
personnel  assigned  to  the  acquisition  programs. 
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Recommendation  Number  One.  The  first  and  most  effective 
recommendation  proposes  a  re-structuring  of  the  existing 
organizational  framework,  specifically  by  elimination  of  the 
current  configuration  and  data  manager  support  organizations 
in  the  matrix  program  offices.  Based  upon  the  research 
findings  of  this  project  (chapter  four),  configuration  and 
data  management  duties  are  regarded  as  essential  for 
successful  program  completion;  however,  utilization 
(uncontrollable)  of  the  configuration  and  data  manager  is 
not  regarded  as  essential.  Organizational  re-structuring 
should  move  the  configuration  and  data  manager  from  the 
configuration  and  data  management  office  physically  into  the 
system  program  office  (Figure  10  and  11).  This  move  will 
increase  their  day-to-day  contact  (familiarity)  with,  and 
their  avai labi 1 i ty  to,  the  program  and  the  program  manager 
and  is  expected  to  increase  the  productivity  and  efficiency 
of  both  the  system  program  office  and  the  configuration  and 
data  management  function.  The  close  physical  proximity  of 
the  program  manager  and  the  configuration  and  data  manager 
is  necessary  and  will  serve  to  facilitate  communication, 
will  help  to  affirm  competence  and  assure  familiarity  with 
the  program,  and  will  allow  a  precise  agreement  on 
expectations  and  responsibilities.  The  "non-essential" 
image  of  the  configuration  and  data  manager  will 
unquestionably  be  diminished  in  an  environment  conducive  to 
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Ourront  Organizational  St  i  mtur 


communication.  Based  upon  the  program  manager's  opportunity 
to  observe  and  receive  direct  input  from  the  available 
configuration  and  data  manager,  the  utilization  rate  of  the 
configuration  and  data  manager  functional  will  assuredly 
increase  . 

Vital  to  the  successful  enactment  of  this  recommendation 
are  the  previously  mentioned  strategies  for  enhanced 
productivity.  These  strategies  must  be  enacted  concurrently 
with  this  recommendation. 

Additionally,  implementation  of  recommendation  o^e  would 
requires  that  the  centralized  training  responsibility  be 
accomplished  by  the  staff  agency.  Trainees  would  be 
assigned  in  such  a  manner  that  existing,  qualified 
configuration  and  data  managers  would  be  delegated  the  duty 
of  orienting  a  trainee  manager.  Following  comprehensive 
training,  the  newly  oriented  trainee  manager  would  he 
transferred  to  a  managerial  vacancy  within  a  system  program 
o*  f i ce . 

Many  of  the  expert  supervisory  personnel  currently  in  the 
configuration  and  data  management  office  of  the  matrix 
program  office  could  be  transferred  to  an  expanded  staff 
agency.  It  will  be  the  responsibility  of  these  experts  to 
create  and  administer  an  expanded  training  program,  to 
standardize  the  con f i gu r a t i on  and  data  management  practices 
and  policies,  to  answer  questions  from  system  program 
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office's  configuration  and  data  managers,  and  to  supplement 
system  program  office's  configuration  and  data  managers  on 
major  activities.  However,  some  of  the  supervisory 
positions  would  be  eliminated  as  configuration  and  data 
management  positions  are  reclassified  to  meet  the  objectives 
of  the  ASD  commander. 

Implementation  of  this  first  recommendation  would 
increase  the  efficiency  and  utilization  of  the  configuration 
and  data  management  duties  in  a  short  period  of  time. 
However,  the  major  problem  with  executing  this 
recommendation  is  the  disruption  of  the  current 
organizational  structure  and  the  loss  of  some  top-level 
positions  for  configuration  and  data  management  personnel. 

Recommendation  Number  Two.  The  second  recommendation 
proposes  enactment  of  the  productivity  enhancement 
strategies  within  the  existing  organizational  structure. 

The  overwhelming  problem  with  this  recommendation  is  that 
the  factors  affecting  utilization  and  availability  will 
neither  be  addressed  adequately  nor  eliminated.  As  stated 
in  the  conclusions  portion  of  this  chapter,  utilization  and 
availability  are  the  principal  factors  which  limit 
productivity  of  the  configuration  and  data  management 
personnel.  Unless  the  configuration  and  data  managers  are 
available  to  and  utilized  by  program  managers,  the 
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consequential  improvements  based  on  those  productivity 
enhancement  strategies  will  have  only  a  minor  effect  on 
increasing  productivity  within  the  matrixed  configuration 
and  data  management  functional  office. 

Suggestions  for  Future  Research 

The  following  areas  are  suggested  for  future  pertinent 
research : 

1.  The  method  used  in  this  research  effort  could  be 
applied  to  a  larger  population  of  configuration  and  data 
managers  in  different  organizational  structures  (functional 
and/or  project).  Perhaps  a  different  environment  will  allow 
for  the  complete  dismissal  of  the  organizational  structure 
as  the  problem  of  decreased  productivity. 

2.  The  method  used  in  this  research  effort  could  be 
applied  to  configuration  and  data  managers  within  commercial 
matrix  structured  organizations,  such  as  a  defense 
contractor  organization.  The  application  of  a  matrix 
organizational  structure  is  certainly  not  unique  to  ASD. 
Problems  similar  to  specialized  competence  must  be 
encountered  by  defense  contractors  when  preparing  to  meet 
program  goals  and  adapt  to  changes  in  existing  government 
contracts . 
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Summary 

In  this  research  effort,  valuable  information  pertaining  to 
the  problem  of  limited  productivity  within  configuration  and 
data  management  was  gained.  The  program  manager's  conscious 
exclusion  (non-utilization)  of  both  knowledgeable  and 
trainee  configuration  and  data  managers  coupled  with  the 
perceived  non-availability  of  the  configuration  and  data 
managers  were  identified  as  the  major  factors  contributing 
to  decreased  productivity.  Two  recommendations  are  proposed 
to  rectify  the  identified  deficiencies  within  the 
configuration  and  data  management  area.  Implementation  of 
the  first  recommendation  will  fully  remedy  the  situational 
deficiencies;  however,  executing  the  second  recommendation 
will  only  partially  increase  productivity.  whether  or  not 
either  of  these  recommendations  is  implemented  will  be 
determined  by  appropriate  personnel.  The  upper  echelon  of 
officers  must  understand  that  the  highest  degree  of 
efficiency  is  only  achieved  by  maximizing  the  availability 
of  and  utilization  of  each  functional  area.  Increased 
productivity  begins  with  appropriate  utilization  of  all 
available  resources.  Program  managers  must  realize  that, 
like  other  functionals,  configuration  and  data  management 
personnel  are  assets  when  properly  utilized. 
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Appendix  A:  Definitions 


Collocated.  A  type  of  assignment  whereby  a  functional 
specialist  is  operationally  assigned  to  a  two-letter  system 
program  organization  to  meet  a  specific  need.  Collocated 
personnel  are  physically  located  with  the  two-letter 
organization  and  are  responsible  through  appropriate 
channels  to  the  organization  deputy.  Whether  resources  are 
further  collocated  to  a  three-letter  organization  is 
determined  by  agreement  between  the  chief  of  the  three- 
letter  organization  and  the  senior  collocate.  Collocation 
of  employees  should  be  considered  whenever  full-time  support 
is  needed  from  an  employee  for  a  period  of  over  90  days. 
(7:2) 

Dedicated.  An  ?!"niny»o  who  is  devoted  to  a  specific  program 
on  a  full-time  basis  as  required  by  the  program  manager. 

Functional  Assignment  (functionals).  An  employee  whose  job 
assignment  is  to  a  given  functional  area.  For  purposes  of 
this  study,  functional  assignment  would  be  an  employee  who 
is  working  as  a  configuration  and  data  manager  (CM/DM). 

Functional  Deputy.  A  deputy  ( ie  ASD/AC,  ASD/EN,  etc.) 
responsible  for  maintaining  a  body  of  expertise  in  a  given 
functional  area  to  be  applied  in  support  of  acquisition 
activities  (11:2). 

Matrixing.  The  concept  of  classifying  and  assigning  skills 
by  functional  area  and  collocating  personnel  with  these 
skills  to  support  program/project  organizations  (7:2). 

Program  Manager.  An  individual  who  is  responsible  for  the 
progress  of  all  facets  of  the  program. 

Specialized  Aptitude:  The  individual's  ability  to 
assimilate  new  information  as  well  as  the  natural  tendency 
to  personal]  . ugage  in  other  than  familiar  areas  and 
issues . 

Specialized  Competence:  "an  individual's  feelings  and 
confidence  about  his  abilities  in  mastering  an 
organizational  and  work  settings"  (34:451). 

Specialized  Training/Education:  Training  and  education 
which  is  specifically  directed  at  a  functional  area. 
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1. 


ipendix  B:  Configuration  and  Data  Manager  Survey  Instrument 


Educati onal  Degree  f  s ) 

What  is  the  highest  level  of  education  you  have  achieved? 

Are  you  presently  enrolled  in  any  educational  courses?  Any 
particular  area  of  specialization? 

2 .  Program  Attributes 

2.1  What  phase  is  your  program  currently  in  (  i.e.,  Concept 
Definition,  Demonstration/Validation,  Full  Scale 
Development,  Production  and  Deployment,  or  Operations 
Support ) ? 

2.2  Are  you  presently  working  on  more  than  one  program? 

2.3  Do  you  have  time  to  do  all  that  needs  to  be  done  in 
CM/DM  area  for  the  program?  If  not,  what  areas  are  being 
neglected? 

3 .  Self-rated  Specialized  Competence 

In  the  discussion  that  follows,  the  term  "specialized 
competence"  will  be  used.  The  meaning  of  that  term  is  the 
ability  to  assimilate  and  use  specialized  information 
related  to  your  specialty.  Here,  it  applies  specifically  to 
your  ability  as  a  Conf iguration  and  Data  Manager  to 
understand  specialized  CM/DM  concepts  at  a  level  of  detail 
commensurate  with  your  management  position  and  to  properly 
factor  that  knowledge  into  your  decision  making.  The 
specialized  competence  you  have  may  be  due  to  a  variety  of 
factors  such  as  education,  experience,  and  your  own 
intrinsic  aptitude.  Please  keep  in  mind  that  the  following 
questions  do  not  deal  with  how  well  you  do  your  job  as  a 
Configuration  and  Data  Manager.  The  focus  is  how 
comfortable  you  are  with  your  own  ability  to  deal  with 
specialized  issues. 
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3.1  How  well  do  you  meet  your  own  personal  expectations  for 
CM/DM  specialized  expertise  in  doing  this  job? 


a.  How  comfortable  do  you  handle  specifications?  What 
baseline  are  you  currently  monitoring?  Do  you  think  that 
the  following  actions  are  something  a  CM/DM  should  be  doing? 
Do  you  think  that  this  task  is  something  the  program  manager 
should  be  doing?  On  your  program,  who  actually  performs 
these  actions? 

Coordinate  specification  review 
Control  file  copies 
Review  specification  for  content 
Review  specification  for  format 

b.  How  comfortable  are  you  working  with  CDRLs?  How  many 
changes  have  you  accomplished  during  the  last  year?  Do  you 
think  that  the  following  actions  are  something  a  CM/DM 
should  be  doing?  Do  you  think  that  this  task  is  something 
the  program  manager  should  be  doing?  On  your  program,  who 
actually  performs  these  actions? 

Make  Data  Call 

Run  Data  Requirements  Review  Board 
Monitor  Data  Submittal 
Approve  Data  Format 

c.  How  do  you  feel  about  handling  the  ECP  process?  How 
many  CCB  have  you  participated  in  during  the  last  year?  Do 
you  think  that  the  following  actions  are  something  a  CM/DM 
should  be  doing?  Do  you  think  that  this  task  is  something 
the  program  manager  should  be  doing?  On  your  program,  who 
actually  performs  these  actions? 

Coordinate  ECP  Review 

Conduct  Technical  Coordinating  Meeting 
Brief  CCB  to  program  manager 

Gather  information  to  support  the  program  manager  CCB  brief 

d.  How  comfortable  do  you  understand  the  aspects  of  a 

conf iguration  audit?  How  many  audits  have  you  attended?  Do 
you  think  that  the  following  actions  are  something  a  CM/DM 
should  be  doing?  Do  you  think  that  this  task  is  something 
the  program  manager  should  be  doing?  On  your  program,  who 
actually  performs  these  actions? 

Coordinate  All  Technical  Arrangements 
Chair  the  Audits 
Record/Track  all  Write-ups 
Sign  the  Minutes 
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e.  How  comfortable  do  you  handle  Status  Accounting?  How  do 
you  track  change  processing?  Do  you  think  that  the 
following  actions  are  something  a  CM/DM  should  be  doing?  Do 
you  think  that  this  task  is  something  the  program  manager 
should  be  doing?  On  your  program.,  who  actually  performs 
these  actions? 

Approve  the  System 

Manage  the  Data 

Focal  point  for  Problems 

f.  How  comfortable  do  you  handle  the  engineering  drawings 
process?  Who  is  the  engineering  data  manager  on  your 
program?  Do  you  think  that  the  following  actions  are 
something  a  CM/DM  should  be  doing?  Do  you  think  that  this 
task  is  something  the  program  manager  should  be  doing?  On 
your  program,  who  actually  performs  these  actions? 

Establish  Control  Requirements 

Review  Preliminary  Drawings 

Chair  Interim  Program  Reviews  (IPR) 

Approve  Drawings 

g.  How  comfortable  do  you  understand  the  regulations 
associated  with  data  rights?  What  level  of  drawings  are 
required  on  your  program?  Do  you  think  that  the  following 
actions  arc  something  a  CM/DM  should  be  doing?  Do  you  think 
that  this  task  is  something  the  program  manager  should  be 
doing?  On  your  program,  who  actually  performs  these 
actions? 

Determine  Contract  Requirements 
Determine  Violations 
Focal  Point  for  Concerns 


3.2  Imagine  that  the  configuration  and  data  manager  had 
apprentices.  To  what  degree  would  the  CM/DM  specialized 
competence  you  exhibit  be  a  good  model  for  your  apprentice? 

3.3  To  what  extent  are  your  talents,  or  the  places  you  can 
concentrate  your  attention,  in  areas  other  than  CM/DM 
aspects  of  the  program.  If  so,  which  functional  area  is 
your  attention  placed  in? 
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3.4  Considering  the  time  you  have  spent  in  your  position, 
how  familiar  are  you  with  the  following  key  CM/DM  aspects  of 
the  program? 

a.  Statement  of  Work 

b.  Special  Contractual  Requirements 

c.  Contract  Data  Requirements  List 

d.  Program  Schedule/hilestones 

3.5  Do  you  feel  you  have  all  the  CM/DM  specialized  skills 
you  need  to  perform  well  in  your  job?  Do  you  have  any  weak 
areas? 

3.6  How  do  you  compare  in  CM/DM  specialized  competence  to: 
other  configuration  and  data  managers  in  your  SPO?;  in 
similar  SPOs?;  in  ASD? 

3.7  How  do  you  compare  in  the  level  of  specialized 
competence  (CM/DM)  with  the  other  functional  experts  on  the 
program? 

3.8  How  much  does  the  program  manager  utilize  your 
specialized  CM/DM  competence?  Do  you  attend  program 
reviews?  If  sn.  which  ones? 


Self-rated 


)titude 


4.1  How  strong  is  your  natural  tendency  to  ask  questions  or 
solicit  information  concerning  non-CM/DM  aspects  of  your 
program?  If  strong  tendency,  which  functionals  do  you 
correspond  with? 


4.2  How  quickly  would  you  say  you  "come  up  to  speed"  as  far 
as  understanding  CM/DM  related  specialized  information  that 
is  new  to  you?  For  example,  how  familiar  are  you  with: 


a.  Do D- STD- 2167 

b.  Engineering  Data  Requirements  File 

c.  Software  Documentation 
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5. 


Assume  you  got  notification  to  leave  this  job  within  sixty 
days  and  were  involved  in  choosing  your  replacement. 

5.1  How  important  a  consideration  do  you  think  it  should  be 
that  your  replacement  have  a  high  level  of  specialized  CM/DM 
competence? 

5.2  Suppose  the  person  chosen  to  replace  you  had  a  great 
deal  of  acquisition  and  management  experience  but  little 
CM/DM  experience  and  had  trouble  grasping  CM/D”  specialized 
issues.  Do  you  think  his  lack  of  specialized  CM/DM  ability 
might  be  a  serious  detriment  to  your  program?  If  so,  at 
what  position  level(s)? 

6 .  Suggestions  for  the  Betterment 

6.1  In  your  opinion,  what  attributes  should  a  configuration 
and  data  manager  (specialist)  possess  in  order  to  perform 
his  job? 

a.  Education  (level) 

b.  Experience  (years) 

c.  Experience  (area) 

d.  Age  (years) 

e.  Sex  (male/female) 

f.  CM/DM  Knowledge 


6.2  In  your  opinion,  the  most  capable  CM/DMs  possesses 
understanding  in  which  functional  background ( s ) ? 

a.  Engineering 

b.  Logistics 

c.  Contracting 

d.  Administration 

e.  Program  Control 

f.  Other 
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6.3  Specialized  CM/DM  Training/Education  Suggestions 


What  kinds  of  specific  specialized  areas  do  you  believe  may 
warrant  increased  emphasis  in  the  training  and  education  of 
personnel  who  will  become  configuration  and  data  managers  of 
the  future? 

a.  Software  Documentation 

b.  Software  languages  (PDL,  Ada,  Cobol,  etc) 

c.  Engineering  drawings  regulations 

d.  Systems  Engineering  Application 

e.  Automated  Data  Bases 

f.  Auditing  CAD/CAM  items 

g.  Software  Independent  Verification/Validation 

h.  Other 
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Appendix  C:  Configuration  and  Data  Manager  Collection  Sheet 


Program:  _ 

Configuration  and  Data  Manager  Name: 

Rank/Grade:  _ 

Program  Ma^-.ger  Name:  _ 

Organi?g  .on:  _ 


1.  Educational  Level  Category: 

A.  Beyond  Bachelor. 

B.  Technical  Bachelor. 

C.  Non-technical  Bachelor. 

D.  Associate  Degree. 

E.  High  School  Diploma. 

Educational  Field 
Technical 
Business 
Sciences 

Comments : 


2-1  Program  Phase  Category: 

A.  Concept  Definition 

B.  Demonstration/Validation 

C.  Full  Scale  Development 

D.  Production 

E.  Operation  Support 

2 . 2  Workload  Rating: 

Rating  Scale : 

1.  Dedicated  to  one  program 

2.  Dedicated  to  more  than  one  program 

2 . 3  Time  Requirements  Rating: 

Rating  Scale: 

1.  Strongly  Agree 

2 .  Agree 

3 .  Neutral 

A  .  Disagree 
5.  Strongly  Disagree 

Comment  s  : 
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Rating: 


3 .  Self-Rated  Specialized  Competence 


3.1.  Personal  Expectations  Rating: 

Rating  Scale: 

00-14.  Not  very  well 
15-19.  Marginally 
20-24.  Adequately 
25-29.  Fairly  well 
30-35.  Extremely  well 

3.1.  a  Spec i f i cat i on  Rating: 

Baseline:  F  A  P  N/A 


CM/DM  PM  Actual: 

Coordinate  specification  review 
Control  file  copies 
Review  specification  for  content 
Review  specification  for  format 

Comments : 


3.1.b  Contract  Data  Requirements  List  Rating: 


Number  of  Changes: 


Make  Data  Call 

Run  Data  Requirements  Review  Board 
Monitor  Data  Submittal 
Approve  Data  Format 


CM/DM 


Actual  : 


Comments  : 
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Rati ng • 


Number  of  CCBs: 


Coordinate  ECP  Review 
Conduct  Technical  Coordinating 
Brief  CCB  to  program  manager 
Gather  information  to  support 
the  program  manager  CCB  brief 


CM/DM 

Meeting 


PM  Actual: 


Comments : 


3.1.d  Configuration  Audits  Rating: 

Number  of  Audits; 


C M/DM  PM  Actual: 

Coordinate  All  arrangements 
Chair  the  Audits 
Record/Track  all  write-ups 
Sign  the  Minutes 

Comments : 


3.1.e  Status  Accounting  Rating: 

Tracking:  Manual  Automated 


CM/DM  PM  Actual: 

Approve  the  System 

Manage  the  Data 

Focal  point  for  Problems 

Comments : 
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3 . 1 .  f  Engineering  Drawings  Rating:  _ 

EDMO:  CM/DM  Staff  PM 

CM/DM  PM  Actual: 

Establish  Control  Requirements 
Review  Preliminary  Drawings 
Chair  Interim  Program  Reviews 
Approve  Drawings 

Comments : 


3.1.g  Data  Rights  Rating: 

Level  of  Drawings:  _ 


CM/DM  PM  Actual: 

Determine  Contract  Requirements 
Determine  Violations 
Focal  Point  for  Concerns 

Comments : 


3 . 2  Model  for  Apprentice 


Rating  Scale: 

00-14.  Poor  Model 
15-19.  Fair  Model 
20-24.  Acceptable  Model 
25-29.  Pretty  Good  Model 
30-35.  Exceptional  Model 


Comments  : 


Rating : 
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3 . 3  Talents  Elsewhere 


Rating 


Rating  Scale: 

00-14.  Far  more  talented  in  other  areas 
15-19.  More  talented  in  other  areas 
20-24.  Balanced 
25-29.  Specialized  leaning 

30-35.  Most  talented  in  specialized  aspects 

Comments : 

a.  Engineering 

b.  Logistics 

c.  Manufacturing 

d.  Contracting 

e.  Other 


3 . 4  Familiarity  Considering  Time 
Rating  Scale: 

00-14.  Not  familiar  at  all 
15-19.  Somewhat  familiar 
20-24.  Familiar  with  key  issues 
25-29.  Quite  familiar  with  key  issues 
30-35.  Very  familiar  with  all  issues 


3. 4. a  Statement  of  Work 


Rating 


Rating: 


Rating  Scale: 

00-14.  Not  familiar  at  all 

15-19.  Somewhat  familiar 

20-24.  Familiar  with  key  issues 

25-29.  Quite  familiar  with  key  issues 

30-35.  Intimately  familiar  with  all  issues 

3.4.b  Special  Contractual  Requirements  Rating: 


3.4.c  Contract  Data  Requirements  List 


3.4.d  Program  Schedule/Mi lestones 


Rating 


Rating: 
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Rating : 


Rating  Scale: 

00-14.  Do  not  have  needed  specialized  skills  at  all 
15-19.  Need  more  specialized  skill  to  perform  well 
20-24.  Have  some  specialized  skills  but  more  would  be 
helpful 

25-29.  Have  most  of  needed  specialized  skills 

30-35.  Have  all  of  the  specialized  skills  needed  to  perform 

well 

Weak  Areas: 


3.6  Comparisons  Rating: 


3. 6.  a  SPO  comparisons  Within  Specialized  Rating: 

Rating  Scale: 

1.  Below  Average  Level 

2.  Average 

3.  Above  Average  Level 

3.6. b  Similar  SPO  comparisons  Rating: 


3.6.c  ASP  comparisons  within  Specialized  Rating: 


3.7  Comparisons  with  other  Functionals  Rating: 

Rating  Scale: 

1.  Below  Average  Level 

2.  Average 

3.  Above  Average  Level 

3.8  Utilization  Factors  Rating: 


Rating  Scale: 

00-14.  Never 
15-19.  Rarely 
20-24.  Occasionally 
25-29.  Frequently 
30-35.  Constantly 

Comments : 
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Rating : 


4 .  Self-Rated  Specialized  Aptitude 
4 . 1  Propensity  to  Question 


Rating 
00-14. 
15-19. 
20-24  . 
25-29  . 
30-35. 


Scale : 

Do  not  seek  information 
Occasionally  inquire  about  issues 
Question  specialized  matters  about  issues 
Tend  to  ask  questions  until  comfortable 
Ask  questions  until  fully  understood 


Comments : 

a.  Contractual 

b.  Engineering 

c.  Logistics 

d.  Manufacturing 

e.  Other 


4.2  Quickness  in  Learning  Rating: 


Rating 

Scale : 

00-14  . 

Do  not  understand  specialized 

matters 

15-19  . 

Not  a  strong  point,  but  will 

understand 

20-24  . 

Average  quickness 

25-29  . 

Above  average  quickness 

30-35  . 

Extremely  quick  in  absorbing 

information 

4. 2. a 

DoD-STD-2 1 6  7 

Rating 

Scale : 

Rating 

00-14  . 

Not  familiar  at  all 

15-19. 

Somewhat  familiar 

20-24  . 

Familiar  with  key  issues 

25-29  . 

Quite  familiar  with  key  issue 

s 

30-35. 

Intimately  familiar  with  all 

i ssues 

4 . 2  .  b 

Enqineerinq  Data  Requirements  File 

Rating 

4 . 2  .  c 

Software  Documentation 

Rating 
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Rating : 


Rating  Scale: 

00-14.  Not  a  factor  at  all 

15-19.  Not  a  very  important  factor 

20-24.  Even  with  other  factors 

25-29.  One  of  the  most  important  factors 

30-35.  Most  important  factor 

5.2  Detriment  Potential  Rating: 


Rating 
00-14  . 
15-19. 
20-24  . 
25-29. 
30-35. 


Scale : 

Would  not  make  a  difference  at  all 

Would  be  unfortunate,  but  not  seriously  harm  program 
Would  be  a  detriment,  but  not  serious 
Would  be  likely  to  hinder  the  program 
Would  seriously  hinder  program 


Position  Level  Entry  Specialist 


Supe  rvi so  r 


6 .  Suggestions 


6 . 1  Attributes 

a.  Education  (level) 

b.  Experience  (years) 

c.  Experience  (area) 

d.  Age  (years) 

e.  Sex  (male/female) 

f.  Job  Knowledge 

Comments : 


Yes  No 


6 . 2  Proficient  CM/DMs 

Unimportant  Helpful  Important 

a.  Engineering 

b.  Logistics 

c.  Contracting 

d.  Administration 

e.  Program  Control 

f.  Other 
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Yes 


No 


a.  Software  Documentation 

b.  Software  languages  (PDL,  Ada,  Cobol,  etc) 

c.  Engineering  drawings  regulations 

d.  Systems  Engineering  Application 

e.  Automated  Data  Bases 

f.  Auditing  CAD/CAM  items 

g.  Software  Independent  Verification/Validation 

h.  Other 
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1. 


In  the  discussion  that  follows,  the  term  "specialized 
competence"  will  be  used.  The  meaning  of  that  term  is  the 
ability  to  assimilate  and  use  specialized  information. 

Here,  it  applies  specifically  to  the  ability  of  the 
Configuration  and  Data  Manager  supporting  your  program  to 
understand  specialized  CM/DM  concepts  at  a  level  of  detail 
commensurate  with  their  position  and  to  properly  apply  that 
knowledge  in  the  support  of  your  decision  making.  Their 
specialized  CM/DM  competence  may  be  due  to  a  variety  of 
factors  such  as  education,  experience,  and  their  own 
intrinsic  aptitude. 

1.1  How  well  does  the  configuration  and  data  manager  meet 
your  expectation  for  the  following  specialized  areas  in 
doing  his  job?  Do  you  think  that  the  following  sub¬ 
activities  are  something  a  CM/DM  should  be  doing?  Do  you 
think  that  the  activities  is  something  you  should  be  doing? 
On  your  program,  who  actually  performs  these  actions? 

a.  Specification  Maintenance 
Coordinate  specif ication  review 
Control  file  copies 

Review  specification  for  content 
Review  specification  for  format 

b.  Contract  Data  Requirements  List 
Make  Data  Call 

Run  Data  Requirements  Review  Board 
Monitor  Data  Submittal 
Approve  Data  Format 

c.  Engineering  Change  Proposals 
Coordinate  ECP  Review 

Conduct  Technical  Coordinating  Meeting 
Brief  CCB  to  program  manager 

Gather  information  to  support  the  program  manager 
CCB  brief 

d.  Configuration  Audits 
Coordinate  All  arrangements 
Chair  the  Audits 
Record/Track  all  write-ups 
Sign  the  Minutes 
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e.  Status  Accounting 
Approve  the  System 
Manage  the  Data 

Focal  point  for  Problems 

f.  Engineering  Drawings 
Establish  Control  Requirements 
Review  Preliminary  Drawings 
Chair  Interim  Program  Reviews 
Approve  Drawings 

g.  Data  Rights 

Determine  Contract  Requirements 
Determine  Violations 
Focal  Point  for  Concerns 

1.2  Suppose  configuration  and  data  manager  had  apprentices. 
Would  the  CM/DM  specialized  competence  exhibited  by  your 
configuration  and  data  manager  make  a  good  model  for  an 
apprentice  to  emulate? 

1.3  To  what  extent  do  you  feel  the  conf iguration  and  data 
manager's  talents,  or  where  he  can  concentrate  his  attention 
best,  are  in  areas  other  than  CM/DM  specialized  aspects  of 
the  program? 

\.A-  Considering  the  time  spent  in  the  job,  do  you  feel  the 
configuration  and  data  manager  is  thoroughly  familiar  with 
the  key  CM/DM  aspects  of  the  program  defined  in  the: 

a.  Statement  of  Work 

b.  Special  Contractual  Requirements 

c.  Contract  Data  Requirements  List 

d.  Program  Schedule/Milestones 

1.5  Do  you  think  the  configuration  and  data  manager  has  all 
the  specialized  CM/DM  skills  he  needs  to  perform  well  in  his 
job?  Does  the  CM/DM  have  any  weak  areas? 

1.6  How  does  the  configuration  and  data  manager  compare  in 
CM/DM  specialized  competence  to  the  previous  configuration 
and  data  manager? 

1.7  Hew  does  the  configuration  and  data  manager  compare  in 
degree  of  specialized  competence  to  other  functional  experts 
on  your  program? 

1.8  How  much  do  you  utilize  your  configuration  and  data 
manager's  specialized  CM/DM  competence?  In  what  areas? 
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2 .  Configuration  and  Data  Manager  Aptitude 

2.1  How  much  do  you  feel  the  configuration  and  data  manager 
has  a  natural  tendency  to  ask  questions  or  solicit 
information  concerning  the  non-CM/DM  aspects  of  the  program? 
If  strong  tendency,  which  areas? 

2.2  How  quickly  does  the  configuration  and  data  manager 
"come  up  to  speed"  as  far  as  understanding  related 
specialized  CM/DM  information  that  is  new  to  him? 

a.  DoD-STD-2167 

b.  Engineering  Data  Requirements  File 

c.  Software  Documentation 

3 .  Importance  of  Specialized  Competence 

Assume  the  configuration  and  data  manager  received 
notification  to  leave  within  sixty  days  and  that  you  were 
involved  in  choosing  his  replacement. 

3.1  How  important  a  consideration  do  you  think  it  should  be 
that  his  replacement  have  a  high  level  of  specialized  CM/DM 
competence? 

3.2  Suppose  the  person  chosen  to  replace  him  had  a  great 
deal  of  acquisition  and  management  experience  but  little 
CM/DM  experience  and  had  trouble  grasping  CM/DM  specialized 
issues.  Do  you  think  his  lack  of  specialized  ability  might 
be  a  serious  detriment  to  the  program?  If  so,  at  what 
position  level? 


no 


4 .  Suggestions  for  the  Betterment 

4.1  In  your  opinion,  what  attributes  should  a  configuration 
and  data  manager  possess  in  order  to  perform  his  job? 

4.2  In  your  opinion,  do  capable  CM/DMs  come  from  any 
specific  functional  background?  If  so,  which  functional? 

4.3  Specialized  Training/  Education  Suggestions 

What  specific  specialized  CM/DM  areas  do  you  believe  may 
warrant  increased  emphasis  in  the  training  and  education  of 
personnel  who  will  become  configuration  and  data  manager  of 
the  future? 

a.  Software  Documentation 

b.  Software  languages  (PDL,  Ada,  Cobol,  etc) 

c.  Engineering  drawings  regulations 

d.  Systems  Engineering  Application 

e.  Automated  Data  Bases 

f.  Auditing  CAD/CAM  items 

g.  Software  Independent  Ve ri f ’ ca ti on/Va 1 i da t i on 

h.  Other 
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Program  Manager  Name: 

Rank/Grade:  _ 

Organization:  _ 


1.  CM/DM  Specialized  Competence  Rating: 


1 . 1 .  Personal  Expectations 


Rating : 


Rating 

Scale : 

00-14  . 

Not  very  well 

15-19. 

Marginally 

20-24  . 

Adequately 

25-29  . 

Fairly  well 

30-35. 

Extremely  well 

1  .  1  .a 

Soeci f ication 

Rating 

Scale : 

00-14  . 

Not  very  well 

15-19  . 

Marginally 

20-24  . 

Adequately 

25-29  . 

Fair’y  well 

30-35. 

Extremely  well 

Rating: 


CM/DM  PM  Actual: 

Coordinate  specification  review 
Control  file  copies 
Review  specification  for  content 
Review  specification  for  format 


Comments : 


1.1. b  Contract  Data  Requirements  List  Rating: 


CM /DM  PM  Actual: 

Make  Data  Call 

Run  Data  Requirements  Review  Board 
Monitor  Data  Submittal 
Approve  Data  Format 

Comments . 
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CM/ DM 


Rating :  _ 

PM  Actual 


Coordinate  ECP  Review 

Conduct  Technical  Coordinating  Meeting 
Brief  CCB  to  program  manager 
Gather  information  to  support 
the  program  manager  CCB  brief 


Comments : 


1 . 1 . d  Configuration  Audits  Rating:  _ 

CM /DM  PM  Actual 

Coordinate  All  arrangements 
Chair  the  Audits 
Record/Track  all  write-ups 
Sign  the  Minutes 

Comments  : 


1.1. e  Status  Accounting  Rating: 

CM/DM  PM  Actual 

Approve  the  System 

Manage  the  Data 

Focal  point  for  Problems 


Comments : 


1  .  1  .  f  Engineering  Drawings  Rating:  _ 

CM  DM  PM  Actual 

Establish  Control  Requirements 
Review  Preliminary  Drawings 
Chair  Interim  Program  Reviews 
Approve  Drawings 


Comments : 


Rating: 


CM/ DM  PM  Actual 

Determine  Contract  Requirements 
Determine  Violations 
Focal  Point  for  Concerns 

Comments : 


1 . 2  Model  for  Apprentice 


Rating 
00-14  . 
15-19  . 
20-24  . 
25-29  . 
30-35. 


Scale  : 

Poor  Model 
Fair  Model 
Acceptable  Model 
Pretty  Good  Model 
Exceptional  Model 


Rating: 


1 . 3  Talents  Elsewhere 


Rating: 


Rating 
00-14  . 
15-19. 
20-24  . 
25-29  . 
30-35. 


Scale  : 

Far  more  talented  in  other  areas 
More  talented  in  other  areas 
Balanced 

Specialized  leaning 

Most  talented  in  specialized  aspects 


1  . 4  Familiarity  Consideri ng  Time 


Rating: 


Rating 
OO-.M. 
15-19. 
20-24  . 
25-29. 
30-35. 


Scale  : 

Not  familiar  at  all 
Somewhat  familiar 
Familiar  with  key  issues 
Quite  familiar  with  key  issues 
Intimately  familiar  with  all  issues 


1 . 4 . a  Statement  of  Work 


Rating 
00-14  . 
15-19  . 
20-24  . 
25-29. 
30-  35  . 


Scale  : 

Not  familiar  at  all 
Somewhat  familiar 
Familiar  with  key  issues 
Quite  familiar  with  key  issues 
Very  familiar  with  all  issues 


Rating: 


1 . 4 . b  Special  Contractual  Requirements  Rating: 


1  14 


1.4.C  Contract  Data  Requirements  List  Rating: 


1.4.d  Program  Schedule/Milestones  Rating: 


1.5  Specialized  CM/DM  Skills  Rating:  _ 

Rating  Scale: 

00-14.  Do  not  have  needed  specialized  skills  at  all 
15-19.  Need  more  specialized  skill  to  perform  well 
20-24.  Have  some  specialized  skills  but  more  would  be 
helpful 

25-29.  Have  most  of  needed  specialized  skills 

30-35.  Have  all  of  the  specialized  skills  needed  to  perform 

well 

Comments : 


1.6  Comparisons  to  Previous  C M/DM  Rating: 

Rating  Scale: 

1.  Below  Average  Level 

2.  Average 

3.  Above  Average  Level 


1.7  Comparisons  to  Other  Functionals  Rating: 

Rating  Scale: 

1.  Below  Average  Level 

2.  Average 

3.  Above  Average  Level 


1.8  Utilization  Factors  Rating: 

Rating  Scale: 

00-14.  Never 
15-19.  Rarely 
20-24.  Occasionally 
25-29.  Frequently 
30-35.  Constantly 

Comments : 
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2. 


Rating: 


2 . 1  Propensity  to  Question 


Rating : 


Rating 
00-14  . 
15-19  . 
20-24  . 
25-29. 
30-35. 


Scale : 

Do  not  seek  information 
Occasionally  inquire  about  issues 
Question  specialized  matters  about  issues 
Tends  to  ask  questions  until  comfortable 
Ask  questions  until  fully  understood 


Comments : 


2.2  Quickness  in  Learning  Rating: 

Rating  Scale: 

00-14.  Do  not  understand  specialized  matters 
15-19.  Not  a  strong  point,  but  will  understand 
20-24.  Average  quickness 
25-29.  Above  average  quickness 

30-35.  Extremely  quick  in  absorbing  information 
Comments : 


2. 2. a  DoD-STD-2167 


Rating 
00-14  . 
15-19. 
20-24  . 
25-29  . 
30-35. 


Scale:  Rating: 

Not  familiar  at  all 
Somewhat  familiar 
Familiar  with  key  issues 
Quite  familiar  with  key  issues 
Intimately  familiar  with  all  issues 


2.2.b  Engineering  Data  Requirements  File  Rating: 


2.2.c  Software  Documentation 


Rating: 
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3. 


3 . 1  Importance 


Rating : 


Rating  Scale: 


00-14. 
15-19. 
20-24  . 
25-29. 
30-35. 


Not  a  factor  at  all 

Not  a  very  important  factor 

Even  with  other  factors 

One  of  the  most  important  factors 

Most  important  factor 


3 . 2  Detriment  Potential 


Rating : 


Rating  Scale: 


00-14  . 
15-19  . 
20-24  . 
25-29  . 
30-35. 


Would  not  make  a  difference  at  all 

Would  be  unfortunate,  but  not  seriously  harm  program 
Would  be  a  detriment,  but  not  serious 
Would  be  likely  to  hinder  the  program 
Would  definitely  seriously  hinder  program 


Position  Level 


Entry  Specialist 


Supe  rvi so  r 
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4. 


4 . 1  Attributes 

a.  Education  (level) 

b.  Experience  (years) 

c.  Experience  (area) 

d.  Age  (years) 

e.  Sex  (male/female) 

f.  Job  Knowledge 

Comments : 


Yes  No 


4 • 2  Proficient  CM/DMs 

Unimportant  Helpful  Important 

a.  Engineering 

b.  Logistics 

c.  Contracting 

d.  Administration 

e.  Program  Control 

f.  Other 


4 • 3  Specialized  CM/DM  Areas/Education 

a.  Software  Documentation 

b.  Software  languages  (PDL,  Ada,  Cobol,  etc) 

c.  Engineering  drawings  regulations 

d.  Systems  Engineering  Application 

e.  Automated  Data  Bases 

f.  Auditing  CAD/CAM  items 

g.  Software  Independent  Verification/Validation 

h.  Other 


Yes 


No 
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Appendix  F:  Competency  Ratings  (Intrinsic  and  Extrinsic) 
Versus  Educational  Background  and  Organizations 


Educational  Background  Comparison 


CM/DM 

PM 

CM/DM 

TECH 

TECH 

ASSOC 

1. 

29.000 

31.000 

26.000 

2  . 

33.000 

29.000 

26 . 000 

3. 

30.000 

31.000 

29.000 

4  . 

28.000 

32.000 

24.000 

5. 

31.000 

28 .000 

34.000 

6  . 

34 . 000 

32.000 

29 .000 

7. 

34.000 

34.000 

28.000 

8  . 

31.000 

30 .000 

27.000 

9. 

34 . 000 

34.000 

10. 


PM 

CM/DM 

PM 

ASSOC 

HIGH 

HIGH 

28.000 

27 .000 

24.000 

27.000 

20 .000 

20 . 000 

27.000 

21 . 000 

20 .000 

27.000 

24 . 000 

24 . 000 

33.000 

27 .000 

24 . 000 

30 .000 

28.000 

23 . 000 

25.000 

20.000 

20.000 

25.000 

29.000 

29 . 000 

25 . 000 

24.000 

26.000 

23.000 

Organizational  Comparison 


CM/DM 

PM 

C  M/DM 

PM 

CM/DM 

PM 

RW 

RW 

YW 

YW 

AE 

AE 

1  . 

29.000 

31.000 

29 . 000 

27.000 

34.000 

34.000 

2. 

33.000 

29.000 

24.000 

27.000 

29 . 000 

29.000 

3. 

30.000 

31 . 000 

27 . 000 

24 . 000 

25.000 

24 .000 

4  . 

28.000 

32 .000 

24.000 

24.000 

29 .000 

30 . 000 

5. 

31.000 

28 .000 

34 . 000 

33.000 

28 .000 

23.000 

6  . 

26.000 

28.000 

28.000 

25.000 

26.000 

23.000 

7  . 

27.000 

24.000 

20 . 000 

20.000 

27 .000 

25.000 

8  . 

26.000 

27.000 

34 . 000 

34 . 000 

31 . 000 

30.000 

9. 

34.000 

32.000 

10. 

20.000 

20 . 000 

11  . 

21.000 

20.000 
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Appendix  G: 


of  Roles  and  Opinions 


Specification 

Intrinsic  Extrinsic 


CM/DM 

PM 

CM/DM 

PM 

Coordinate  specification  review 

24 

03 

27 

00 

Control  file  copies 

25 

02 

27 

00 

Review  specification  for  content 

13 

14 

03 

24 

Review  specification  for  format 

27 

00 

27 

00 

Contract  Data  Reauirements  List 

Intrinsic 

Extrinsic 

CM/DM 

PM 

CM/DM 

PM 

Make  Data  Call 

27 

00 

27 

00 

Run  Data  Requirements  Review  Board 

14 

13 

20 

07 

Monitor  Data  Submittal 

27 

00 

27 

00 

Approve  Data  Format 

27 

00 

27 

00 

Enqineerinq  Chanqe  Proposals 

Intrinsic 

Extrinsic 

CM/DM 

PM 

CM/DM 

PM 

Coordinate  ECP  Review 

15 

12 

08 

19 

Conduct  Tech  Coordinating  Meeting 

13 

14 

16 

11 

Brief  CCB  to  program  manager 

18 

09 

20 

07 

Gather  information  to  support 
the  program  manager  CCB  brief 

27 

00 

27 

00 

Conf iquration  Audits 

Intrinsic 

Extrinsic 

CM/DM 

PM 

CM/DM 

PM 

Coordinate  All  arrangements 

22 

05 

24 

03 

Chair  the  Audits 

25 

02 

20 

07 

Record/Track  all  write-ups 

27 

00 

27 

00 

Sign  the  Minutes 

17 

08 

20 

07 

Status  Accountinq 

Intrinsic 

Extrinsic 

C  M/DM 

PM 

C  M/DM 

PM 

Approve  the  System 

27 

00 

27 

00 

Manage  the  Data 

27 

00 

27 

00 

Focal  point  for  Problems 

24 

03 

13 

14 
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Engineering  Drawings 


Establish  Control  Requirements 
Review  Preliminary  Drawings 
Chair  Interim  Program  Reviews 
Approve  Drawings 

Data  Rights 


Determine  Contract  Requirements 
Determine  Violations 
Focal  Point  for  Concerns 

Suggestions 

Attributes 


Intrinsic  Extrinsic 


CM/DM 

PM 

CM/DM 

PM 

24 

03 

25 

02 

25 

02 

25 

02 

06 

21 

00 

27 

Not 

Applicable 

Intrinsic 

Extrinsic 

CM/DM 

PM 

CM/DM 

PM 

24 

03 

26 

01 

10 

17 

08 

19 

27 

00 

27 

00 

CM/DM 

PM 

a . 

Education  (level) 

Technical 

12 

23 

Associate 

06 

04 

High  School 

09 

00 

b. 

Experience  (years) 

0-2 

04 

02 

2-5 

10 

15 

5  + 

13 

10 

c . 

Experience  (area) 

Engineering 

12 

21 

Logistics 

13 

04 

Manufacturing 

02 

02 

d . 

Age  (years) 

18-30 

Not  a 

factor 

31-40 

50  + 

e . 

Sex  (male/female) 

Male 

Not  a 

factor 

Female 
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Configuration  and  Data  Manager's  Responses 


Unimportant 

Helpful 

ImDor cant 

a . 

Engineering 

03 

09 

15 

b . 

Logistics 

05 

10 

12 

c . 

Contracting 

12 

13 

02 

d. 

Administration 

17 

08 

02 

e . 

Program  Control 

20 

04 

03 

Pro 

gram  Manager's  Responses 

Unimportant 

Helpful 

Impo  r  tant 

a . 

Engineering 

00 

03 

24 

b. 

Logistics 

00 

14 

13 

c . 

Contracting 

15 

07 

05 

d . 

Administration 

13 

08 

09 

e . 

Program  Control 

18 

03 

06 

Specialized  CM/DM  Areas/Education 


Configuration  and  Data  Manager's  Responses 

a.  Software  Documentation 

Yes 

25 

No 

02 

b. 

Software  languages  (PDL,  Ada,  Cobol ,  etc) 

10 

17 

c . 

Engineering  drawings  regulations 

26 

01 

d. 

Systems  Engineering  Application 

24 

03 

e  . 

Automated  Data  Bases 

22 

05 

f  . 

Auditing  CAD/CAM  items 

05 

22 

9- 

Software  Independent  Verification/Validation 

04 

23 

Program  Manager's  Responses 

a.  Software  Documentation 

Yes 

22 

No 

05 

b. 

Software  languages  (PDL,  Ada,  Cobol,  etc) 

00 

27 

c . 

Engineering  drawings  regulations 

20 

07 

d. 

Systems  Engineering  Application 

25 

02 

e  . 

Automated  Data  Bases 

27 

00 

f . 

Auditing  CAD/CAM  items 

12 

15 

9- 

Software  Independent  Verification/Validation 

00 

27 
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